Utility to generate test queries

Usage

  python -m dorieh.platform.dbt.create_test --script SCRIPT [SCRIPT ...]
        [--table TABLE]
        [--db DB]
        [--connection CONNECTION]
        [--autocommit]
        [-h]
        [--verbose]

Options:

Option	Alias	Description
--help	-h	Show this help message and exit
--script SCRIPT [SCRIPT...]	-s SCRIPT [SCRIPT...]	Path to the file to write the test scripts
--autocommit		Use autocommit, default: False
--database DB	--db DB	Path to a database connection parameters file, default: database.ini (in the working directory
--connection CONNECTION	--connection_name CONNECTION	Section in the database connection parameters file
--table TABLE	-t TABLE	Name of the table to being tested, default: None
--verbose		Verbose output, default: False

Details

.. py:module:: dorieh.platform.dbt.create_test

A utility to generate a test based on a sample table.

The tool introspects a table, given as an input and generates a set of queries each of which tests that the data in a certain column has not changed.

The queries are output into a file that can be executed as a single SQL query producing a table with the following columns:

1. Name of the column being tested

2. What value is being tested not to change: MD5 hash, number of distinct records, mean value or variance

3. Whether the value has changed (indicated by string failed) or remained the same (indicated by string passed)

Individual queries are separated by a comment strings:

• -- Test case end

• -- Test case start

so a test runner can execute them individually if desired

.. py:class:: CType(value) :module: dorieh.platform.dbt.create_test

An enumeration.

.. py:attribute:: CType.categorical :module: dorieh.platform.dbt.create_test :value: ‘categorical’

.. py:attribute:: CType.text :module: dorieh.platform.dbt.create_test :value: ‘text’

.. py:attribute:: CType.integral :module: dorieh.platform.dbt.create_test :value: ‘integral’

.. py:attribute:: CType.numeric :module: dorieh.platform.dbt.create_test :value: ‘numeric’

.. py:attribute:: CType.date :module: dorieh.platform.dbt.create_test :value: ‘date’

.. py:class:: Column(name: str, ctype: ~dorieh.platform.dbt.create_test.CType, is_indexed: bool) :module: dorieh.platform.dbt.create_test

.. py:class:: TableFingerprint(context: ~typing.Optional[~dorieh.platform.dbt.dbt_config.DBTConfig] = None) :module: dorieh.platform.dbt.create_test

.. py:attribute:: TableFingerprint.CATEGORICAL_THRESHOLD :module: dorieh.platform.dbt.create_test :value: 24

.. py:method:: TableFingerprint.get_columns() :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.get_categories() -> ~typing.List[~dorieh.platform.dbt.create_test.Column] :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.generate_tests() :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.test_column(c: ~dorieh.platform.dbt.create_test.Column) :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.test_case_sql(name: str, test: str, condition: str) -> str :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.test_exact(sql: str, name: str, test: str) -> str :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.test_approximate(sql: str, name: str, test: str) -> str :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.union() :module: dorieh.platform.dbt.create_test

.. py:method:: TableFingerprint.write_test_script() :module: dorieh.platform.dbt.create_test