Skip to content

Crawler

Source code in docs_src/capo_glue/types/crawler.py
class Crawler(TypedDict, closed=True):
    name: NotRequired["capo_glue.types.name_string.NameString"]
    """<p>The name of the crawler.</p>"""
    role: NotRequired["capo_glue.types.role.Role"]
    """<p>The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.</p>"""
    targets: NotRequired["capo_glue.types.crawler_targets.CrawlerTargets"]
    """<p>A collection of targets to crawl.</p>"""
    database_name: NotRequired["capo_glue.types.database_name.DatabaseName"]
    """<p>The name of the database in which the crawler's output is stored.</p>"""
    description: NotRequired["capo_glue.types.description_string.DescriptionString"]
    """<p>A description of the crawler.</p>"""
    classifiers: NotRequired["capo_glue.types.classifier_name_list.ClassifierNameList"]
    """<p>A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.</p>"""
    recrawl_policy: NotRequired["capo_glue.types.recrawl_policy.RecrawlPolicy"]
    """<p>A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.</p>"""
    schema_change_policy: NotRequired[
        "capo_glue.types.schema_change_policy.SchemaChangePolicy"
    ]
    """<p>The policy that specifies update and delete behaviors for the crawler.</p>"""
    lineage_configuration: NotRequired[
        "capo_glue.types.lineage_configuration.LineageConfiguration"
    ]
    """<p>A configuration that specifies whether data lineage is enabled for the crawler.</p>"""
    state: NotRequired["capo_glue.types.crawler_state.CrawlerState"]
    """<p>Indicates whether the crawler is running, or whether a run is pending.</p>"""
    table_prefix: NotRequired["capo_glue.types.table_prefix.TablePrefix"]
    """<p>The prefix added to the names of tables that are created.</p>"""
    schedule: NotRequired["capo_glue.types.schedule.Schedule"]
    """<p>For scheduled crawlers, the schedule when the crawler runs.</p>"""
    crawl_elapsed_time: "capo_glue.types.milliseconds_count.MillisecondsCount"
    """<p>If the crawler is running, contains the total time elapsed since the last crawl began.</p>"""
    creation_time: NotRequired["capo_glue.types.timestamp.Timestamp"]
    """<p>The time that the crawler was created.</p>"""
    last_updated: NotRequired["capo_glue.types.timestamp.Timestamp"]
    """<p>The time that the crawler was last updated.</p>"""
    last_crawl: NotRequired["capo_glue.types.last_crawl_info.LastCrawlInfo"]
    """<p>The status of the last crawl, and potentially error information if an error occurred.</p>"""
    version: "capo_glue.types.version_id.VersionId"
    """<p>The version of the crawler.</p>"""
    configuration: NotRequired[
        "capo_glue.types.crawler_configuration.CrawlerConfiguration"
    ]
    """<p>Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see <a href="https://docs.aws.amazon.com/glue/latest/dg/crawler-configuration.html">Setting crawler configuration options</a>.</p>"""
    crawler_security_configuration: NotRequired[
        "capo_glue.types.crawler_security_configuration.CrawlerSecurityConfiguration"
    ]
    """<p>The name of the <code>SecurityConfiguration</code> structure to be used by this crawler.</p>"""
    lake_formation_configuration: NotRequired[
        "capo_glue.types.lake_formation_configuration.LakeFormationConfiguration"
    ]
    """<p>Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.</p>"""
    catalog_id: NotRequired["capo_glue.types.catalog_id_string.CatalogIdString"]
    """<p>The ID of the Data Catalog in which the crawler's output is stored.</p>"""

catalog_id instance-attribute

catalog_id: NotRequired[CatalogIdString]

The ID of the Data Catalog in which the crawler's output is stored.

classifiers instance-attribute

classifiers: NotRequired[ClassifierNameList]

A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.

configuration instance-attribute

configuration: NotRequired[CrawlerConfiguration]

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.

crawl_elapsed_time instance-attribute

crawl_elapsed_time: MillisecondsCount

If the crawler is running, contains the total time elapsed since the last crawl began.

crawler_security_configuration instance-attribute

crawler_security_configuration: NotRequired[CrawlerSecurityConfiguration]

The name of the SecurityConfiguration structure to be used by this crawler.

creation_time instance-attribute

creation_time: NotRequired[Timestamp]

The time that the crawler was created.

database_name instance-attribute

database_name: NotRequired[DatabaseName]

The name of the database in which the crawler's output is stored.

description instance-attribute

description: NotRequired[DescriptionString]

A description of the crawler.

lake_formation_configuration instance-attribute

lake_formation_configuration: NotRequired[LakeFormationConfiguration]

Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.

last_crawl instance-attribute

last_crawl: NotRequired[LastCrawlInfo]

The status of the last crawl, and potentially error information if an error occurred.

last_updated instance-attribute

last_updated: NotRequired[Timestamp]

The time that the crawler was last updated.

lineage_configuration instance-attribute

lineage_configuration: NotRequired[LineageConfiguration]

A configuration that specifies whether data lineage is enabled for the crawler.

name instance-attribute

name: NotRequired[NameString]

The name of the crawler.

recrawl_policy instance-attribute

recrawl_policy: NotRequired[RecrawlPolicy]

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

role instance-attribute

role: NotRequired[Role]

The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

schedule instance-attribute

schedule: NotRequired[Schedule]

For scheduled crawlers, the schedule when the crawler runs.

schema_change_policy instance-attribute

schema_change_policy: NotRequired[SchemaChangePolicy]

The policy that specifies update and delete behaviors for the crawler.

state instance-attribute

state: NotRequired[CrawlerState]

Indicates whether the crawler is running, or whether a run is pending.

table_prefix instance-attribute

table_prefix: NotRequired[TablePrefix]

The prefix added to the names of tables that are created.

targets instance-attribute

targets: NotRequired[CrawlerTargets]

A collection of targets to crawl.

version instance-attribute

version: VersionId

The version of the crawler.