DataQualityGlueTable
¶
Source code in docs_src/capo_glue/types/data_quality_glue_table.py
additional_options
instance-attribute
¶
additional_options: NotRequired[GlueTableAdditionalOptions]
Additional options for the table. Currently there are two keys supported:
-
pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset. -
catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.
catalog_id
instance-attribute
¶
catalog_id: NotRequired[NameString]
A unique identifier for the Glue Data Catalog.
connection_name
instance-attribute
¶
connection_name: NotRequired[NameString]
The name of the connection to the Glue Data Catalog.
database_name
instance-attribute
¶
database_name: NameString
A database name in the Glue Data Catalog.
pre_processing_query
instance-attribute
¶
pre_processing_query: NotRequired[PreProcessingQueryString]
SQL Query of SparkSQL format that can be used to pre-process the data for the table in Glue Data Catalog, before running the Data Quality Operation.