Crawler
¶
Source code in docs_src/capo_glue/types/crawler.py
catalog_id
instance-attribute
¶
catalog_id: NotRequired[CatalogIdString]
The ID of the Data Catalog in which the crawler's output is stored.
classifiers
instance-attribute
¶
classifiers: NotRequired[ClassifierNameList]
A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.
configuration
instance-attribute
¶
configuration: NotRequired[CrawlerConfiguration]
Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.
crawl_elapsed_time
instance-attribute
¶
crawl_elapsed_time: MillisecondsCount
If the crawler is running, contains the total time elapsed since the last crawl began.
crawler_security_configuration
instance-attribute
¶
crawler_security_configuration: NotRequired[CrawlerSecurityConfiguration]
The name of the SecurityConfiguration structure to be used by this crawler.
creation_time
instance-attribute
¶
creation_time: NotRequired[Timestamp]
The time that the crawler was created.
database_name
instance-attribute
¶
database_name: NotRequired[DatabaseName]
The name of the database in which the crawler's output is stored.
description
instance-attribute
¶
description: NotRequired[DescriptionString]
A description of the crawler.
lake_formation_configuration
instance-attribute
¶
lake_formation_configuration: NotRequired[LakeFormationConfiguration]
Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.
last_crawl
instance-attribute
¶
last_crawl: NotRequired[LastCrawlInfo]
The status of the last crawl, and potentially error information if an error occurred.
last_updated
instance-attribute
¶
last_updated: NotRequired[Timestamp]
The time that the crawler was last updated.
lineage_configuration
instance-attribute
¶
lineage_configuration: NotRequired[LineageConfiguration]
A configuration that specifies whether data lineage is enabled for the crawler.
recrawl_policy
instance-attribute
¶
recrawl_policy: NotRequired[RecrawlPolicy]
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.
role
instance-attribute
¶
role: NotRequired[Role]
The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.
schedule
instance-attribute
¶
schedule: NotRequired[Schedule]
For scheduled crawlers, the schedule when the crawler runs.
schema_change_policy
instance-attribute
¶
schema_change_policy: NotRequired[SchemaChangePolicy]
The policy that specifies update and delete behaviors for the crawler.
state
instance-attribute
¶
state: NotRequired[CrawlerState]
Indicates whether the crawler is running, or whether a run is pending.
table_prefix
instance-attribute
¶
table_prefix: NotRequired[TablePrefix]
The prefix added to the names of tables that are created.