Skip to content

SeedUrlConfiguration

Source code in docs_src/capo_kendra/types/seed_url_configuration.py 
14
15
16
17
18
class SeedUrlConfiguration(TypedDict, closed=True):
    seed_urls: "capo_kendra.types.seed_url_list.SeedUrlList"
    """<p>The list of seed or starting point URLs of the websites you want to crawl.</p> <p>The list can include a maximum of 100 seed URLs.</p>"""
    web_crawler_mode: NotRequired["capo_kendra.types.web_crawler_mode.WebCrawlerMode"]
    """<p>You can choose one of the following modes:</p> <ul> <li> <p> <code>HOST_ONLY</code>—crawl only the website host names. For example, if the seed URL is "abc.example.com", then only URLs with host name "abc.example.com" are crawled.</p> </li> <li> <p> <code>SUBDOMAINS</code>—crawl the website host names with subdomains. For example, if the seed URL is "abc.example.com", then "a.abc.example.com" and "b.abc.example.com" are also crawled.</p> </li> <li> <p> <code>EVERYTHING</code>—crawl the website host names with subdomains and other domains that the web pages link to.</p> </li> </ul> <p>The default mode is set to <code>HOST_ONLY</code>.</p>"""

seed_urls instance-attribute 

seed_urls: SeedUrlList

The list of seed or starting point URLs of the websites you want to crawl.

The list can include a maximum of 100 seed URLs.

web_crawler_mode instance-attribute 

web_crawler_mode: NotRequired[WebCrawlerMode]

You can choose one of the following modes:

  • HOST_ONLY—crawl only the website host names. For example, if the seed URL is "abc.example.com", then only URLs with host name "abc.example.com" are crawled.

  • SUBDOMAINS—crawl the website host names with subdomains. For example, if the seed URL is "abc.example.com", then "a.abc.example.com" and "b.abc.example.com" are also crawled.

  • EVERYTHING—crawl the website host names with subdomains and other domains that the web pages link to.

The default mode is set to HOST_ONLY.