Skip to content

InvokeInferenceRouterOutput

Source code in docs_src/capo_sagemaker_inference_router_runtime/types/invoke_inference_router_output.py
class InvokeInferenceRouterOutput(TypedDict, closed=True):
    body: NotRequired[
        "capo_sagemaker_inference_router_runtime.types.inference_router_body_blob.InferenceRouterBodyBlob"
    ]
    """<p>The inference that the model returned.</p>"""
    content_type: NotRequired["str"]
    """<p>The MIME type of the inference returned in the response body.</p>"""
    invoked_routed_resource_arn: NotRequired["str"]
    """<p>The Amazon Resource Name (ARN) of the endpoint or inference component that served the request.</p>"""
    routing_reason: NotRequired["str"]
    """<p>The reason that the router chose the destination that served the request. Possible values:</p> <ul> <li> <p> <code>local-preferred</code> – The destination is a healthy destination in the router's Region.</p> </li> <li> <p> <code>capacity-available</code> – No destination in the router's Region could take the request, and two or more destinations in other Regions could.</p> </li> <li> <p> <code>only-healthy</code> – No destination in the router's Region could take the request, and exactly one destination in another Region could.</p> </li> <li> <p> <code>failover</code> – The router moved the request away from destinations in its Region because they were overloaded or returning errors. You also see this value when the router can't assess destination health, and when no destination is healthy and the router tries one anyway.</p> </li> </ul>"""
    invoked_production_variant: NotRequired["str"]
    """<p>The production variant of the destination endpoint that served the request.</p>"""
    custom_attributes: NotRequired[
        "capo_sagemaker_inference_router_runtime.types.custom_attributes_header.CustomAttributesHeader"
    ]
    """<p>Provides additional information in the response about the inference. This is the value that the model container of the destination returned, unchanged.</p>"""

body instance-attribute

body: NotRequired[InferenceRouterBodyBlob]

The inference that the model returned.

content_type instance-attribute

content_type: NotRequired[str]

The MIME type of the inference returned in the response body.

custom_attributes instance-attribute

custom_attributes: NotRequired[CustomAttributesHeader]

Provides additional information in the response about the inference. This is the value that the model container of the destination returned, unchanged.

invoked_production_variant instance-attribute

invoked_production_variant: NotRequired[str]

The production variant of the destination endpoint that served the request.

invoked_routed_resource_arn instance-attribute

invoked_routed_resource_arn: NotRequired[str]

The Amazon Resource Name (ARN) of the endpoint or inference component that served the request.

routing_reason instance-attribute

routing_reason: NotRequired[str]

The reason that the router chose the destination that served the request. Possible values:

  • local-preferred – The destination is a healthy destination in the router's Region.

  • capacity-available – No destination in the router's Region could take the request, and two or more destinations in other Regions could.

  • only-healthy – No destination in the router's Region could take the request, and exactly one destination in another Region could.

  • failover – The router moved the request away from destinations in its Region because they were overloaded or returning errors. You also see this value when the router can't assess destination health, and when no destination is healthy and the router tries one anyway.