Skip to content

DataQualityGlueTable

Source code in docs_src/capo_glue/types/data_quality_glue_table.py
class DataQualityGlueTable(TypedDict, closed=True):
    database_name: "capo_glue.types.name_string.NameString"
    """<p>A database name in the Glue Data Catalog.</p>"""
    table_name: "capo_glue.types.name_string.NameString"
    """<p>A table name in the Glue Data Catalog.</p>"""
    catalog_id: NotRequired["capo_glue.types.name_string.NameString"]
    """<p>A unique identifier for the Glue Data Catalog.</p>"""
    connection_name: NotRequired["capo_glue.types.name_string.NameString"]
    """<p>The name of the connection to the Glue Data Catalog.</p>"""
    additional_options: NotRequired[
        "capo_glue.types.glue_table_additional_options.GlueTableAdditionalOptions"
    ]
    """<p>Additional options for the table. Currently there are two keys supported:</p> <ul> <li> <p> <code>pushDownPredicate</code>: to filter on partitions without having to list and read all the files in your dataset.</p> </li> <li> <p> <code>catalogPartitionPredicate</code>: to use server-side partition pruning using partition indexes in the Glue Data Catalog.</p> </li> </ul>"""
    pre_processing_query: NotRequired[
        "capo_glue.types.pre_processing_query_string.PreProcessingQueryString"
    ]
    """<p>SQL Query of SparkSQL format that can be used to pre-process the data for the table in Glue Data Catalog, before running the Data Quality Operation.</p>"""

additional_options instance-attribute

additional_options: NotRequired[GlueTableAdditionalOptions]

Additional options for the table. Currently there are two keys supported:

  • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

  • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

catalog_id instance-attribute

catalog_id: NotRequired[NameString]

A unique identifier for the Glue Data Catalog.

connection_name instance-attribute

connection_name: NotRequired[NameString]

The name of the connection to the Glue Data Catalog.

database_name instance-attribute

database_name: NameString

A database name in the Glue Data Catalog.

pre_processing_query instance-attribute

pre_processing_query: NotRequired[PreProcessingQueryString]

SQL Query of SparkSQL format that can be used to pre-process the data for the table in Glue Data Catalog, before running the Data Quality Operation.

table_name instance-attribute

table_name: NameString

A table name in the Glue Data Catalog.