Skip to content

InvokeInferenceRouterAsyncInput

Source code in docs_src/capo_sagemaker_inference_router_runtime/types/invoke_inference_router_async_input.py
class InvokeInferenceRouterAsyncInput(TypedDict, closed=True):
    inference_router_name: "str"
    """<p>The name of the inference router to invoke.</p>"""
    routing_target_name: "str"
    """<p>The name of the routing target in the inference router. The routing target must be configured for asynchronous inference.</p>"""
    content_type: NotRequired["str"]
    """<p>The MIME type of the input data in Amazon S3. The router passes it to the destination.</p>"""
    accept: NotRequired["str"]
    """<p>The MIME type that you want for the inference response. The router passes it to the destination.</p>"""
    custom_attributes: NotRequired[
        "capo_sagemaker_inference_router_runtime.types.custom_attributes_header.CustomAttributesHeader"
    ]
    """<p>Provides additional information about the inference request. The value is an opaque string that the router passes, unchanged, to the model container of the destination that serves the request. The value can contain up to 1,024 visible US-ASCII characters.</p>"""
    inference_id: NotRequired["str"]
    """<p>An identifier for the inference request. If you don't provide one, SageMaker AI generates one and returns it in the response.</p>"""
    input_location: "str"
    """<p>The Amazon S3 URI of the object that contains the input data for the inference. The bucket must be in the inference router's Amazon Web Services Region and owned by the router's account. The destination endpoint's execution role reads the object, and the destination's asynchronous inference limits apply to it.</p>"""
    s3_output_path_extension: NotRequired["str"]
    """<p>The path extension that is appended to the routing target's Amazon S3 output path where the inference response is stored.</p>"""
    filename: NotRequired["str"]
    """<p>The file name of the inference response stored in Amazon S3. If you don't provide one, SageMaker AI generates one.</p>"""
    request_ttl_seconds: NotRequired["int"]
    """<p>The time to live of the request, in seconds, measured from when the router accepts it. If no destination starts the request in this time, the request fails. The default is 21,600 seconds (6 hours).</p>"""
    invocation_timeout_seconds: NotRequired["int"]
    """<p>The maximum time, in seconds, that a destination can spend processing the request before the request is marked as expired. The default is 900 seconds (15 minutes).</p>"""

accept instance-attribute

accept: NotRequired[str]

The MIME type that you want for the inference response. The router passes it to the destination.

content_type instance-attribute

content_type: NotRequired[str]

The MIME type of the input data in Amazon S3. The router passes it to the destination.

custom_attributes instance-attribute

custom_attributes: NotRequired[CustomAttributesHeader]

Provides additional information about the inference request. The value is an opaque string that the router passes, unchanged, to the model container of the destination that serves the request. The value can contain up to 1,024 visible US-ASCII characters.

filename instance-attribute

filename: NotRequired[str]

The file name of the inference response stored in Amazon S3. If you don't provide one, SageMaker AI generates one.

inference_id instance-attribute

inference_id: NotRequired[str]

An identifier for the inference request. If you don't provide one, SageMaker AI generates one and returns it in the response.

inference_router_name instance-attribute

inference_router_name: str

The name of the inference router to invoke.

input_location instance-attribute

input_location: str

The Amazon S3 URI of the object that contains the input data for the inference. The bucket must be in the inference router's Amazon Web Services Region and owned by the router's account. The destination endpoint's execution role reads the object, and the destination's asynchronous inference limits apply to it.

invocation_timeout_seconds instance-attribute

invocation_timeout_seconds: NotRequired[int]

The maximum time, in seconds, that a destination can spend processing the request before the request is marked as expired. The default is 900 seconds (15 minutes).

request_ttl_seconds instance-attribute

request_ttl_seconds: NotRequired[int]

The time to live of the request, in seconds, measured from when the router accepts it. If no destination starts the request in this time, the request fails. The default is 21,600 seconds (6 hours).

routing_target_name instance-attribute

routing_target_name: str

The name of the routing target in the inference router. The routing target must be configured for asynchronous inference.

s3_output_path_extension instance-attribute

s3_output_path_extension: NotRequired[str]

The path extension that is appended to the routing target's Amazon S3 output path where the inference response is stored.