The project_loader Module

This module is a command line tool to introspect and ingest into a database a directory, containing CSV (or CSV-like, e.g. FST, JSON, SAS, etc.) files. The directory can be structured, e.g. have nested subdirectories. All files matching a certain name pattern at any nested subdirectory level are included in the data set.

In the database, a schema is crated based on the given project name. For each file in the data set a table is created. The name of the table is constructed from the relative path of the incoming data file with OS path separators (e.g. ‘/’) being replaced with underscores (‘_’).

is_relative_to(p: PurePath, *other)[source]

Return True if the path is relative to another path or False.

remove_ext(p: str) str[source]
class ProjectLoader(context: Optional[LoaderConfig] = None)[source]

Class, implementing Project Loading functionality.

Parameters

context – Configuration options, usually provided as command line arguments

classmethod new_domain(name: str, file_path: str)[source]
introspect(entry: str)[source]
save_registry()[source]
run()[source]

Run introspection and ingestion.

Returns

nothing