Skip to content

ScalingPolicyMetric

Source code in docs_src/capo_sagemaker/types/scaling_policy_metric.py 
11
12
13
14
15
class ScalingPolicyMetric(TypedDict, closed=True):
    invocations_per_instance: NotRequired["capo_sagemaker.types.integer.Integer"]
    """<p>The number of invocations sent to a model, normalized by <code>InstanceCount</code> in each ProductionVariant. <code>1/numberOfInstances</code> is sent as the value on each request, where <code>numberOfInstances</code> is the number of active instances for the ProductionVariant behind the endpoint at the time of the request.</p>"""
    model_latency: NotRequired["capo_sagemaker.types.integer.Integer"]
    """<p>The interval of time taken by a model to respond as viewed from SageMaker. This interval includes the local communication times taken to send the request and to fetch the response from the container of a model and the time taken to complete the inference in the container.</p>"""

invocations_per_instance instance-attribute 

invocations_per_instance: NotRequired[Integer]

The number of invocations sent to a model, normalized by InstanceCount in each ProductionVariant. 1/numberOfInstances is sent as the value on each request, where numberOfInstances is the number of active instances for the ProductionVariant behind the endpoint at the time of the request.

model_latency instance-attribute 

model_latency: NotRequired[Integer]

The interval of time taken by a model to respond as viewed from SageMaker. This interval includes the local communication times taken to send the request and to fetch the response from the container of a model and the time taken to complete the inference in the container.