Skip to content

AsyncInferenceClientConfig

Source code in docs_src/capo_sagemaker/types/async_inference_client_config.py 
11
12
13
14
15
class AsyncInferenceClientConfig(TypedDict, closed=True):
    max_concurrent_invocations_per_instance: NotRequired[
        "capo_sagemaker.types.max_concurrent_invocations_per_instance.MaxConcurrentInvocationsPerInstance"
    ]
    """<p>The maximum number of concurrent requests sent by the SageMaker client to the model container. If no value is provided, SageMaker chooses an optimal value.</p>"""

max_concurrent_invocations_per_instance instance-attribute 

max_concurrent_invocations_per_instance: NotRequired[
    MaxConcurrentInvocationsPerInstance
]

The maximum number of concurrent requests sent by the SageMaker client to the model container. If no value is provided, SageMaker chooses an optimal value.