The utils Module
API used by data modelling and data loading utilities
- split(node) -> (<class 'str'>, <class 'dict'>)[source]
Given a node in an array produced from JSON/YAML structure returns a name and definition associated with the node
- Parameters
node¶ – an array node, usually from YAML or JSON file
- Returns
a tuple consisting of a name and an array containing definitions
- basename(table)[source]
Given fully qualified table name (schema.table) returns just a basename of a table
- Parameters
table¶ – a fully qualified table name
- Returns
just the base name of the table, a piece of name after the last dot
- entry_to_path(entry: Any) str[source]
Returns valid path for an archive entry
- Parameters
entry¶ – an archive entry or a path to file on the file system
- Returns
path within the arcjve or file system
- class CSVLikeJsonReader(path: str, columns: List, returns_mapping=False)[source]
Class, providing CSV Reader interface for JSON files.
Helps abstracting reading different file types.
- class DataReader(path: str, buffer_size=None, quoting=None, has_header=None, columns=None, delimiter=None)[source]
Generalized reader for columns-structured files, such as CSV, FST and sas7bdat
This class is Context manager and can be used with with block
Example:
- with DataReader(file_path) as reader:
- for row in reader.rows():
print(row)