Skip to content

InferenceComponentRollingUpdatePolicy

Source code in docs_src/capo_sagemaker/types/inference_component_rolling_update_policy.py 
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class InferenceComponentRollingUpdatePolicy(TypedDict, closed=True):
    maximum_batch_size: NotRequired[
        "capo_sagemaker.types.inference_component_capacity_size.InferenceComponentCapacitySize"
    ]
    """<p>The batch size for each rolling step in the deployment process. For each step, SageMaker AI provisions capacity on the new endpoint fleet, routes traffic to that fleet, and terminates capacity on the old endpoint fleet. The value must be between 5% to 50% of the copy count of the inference component.</p>"""
    wait_interval_in_seconds: NotRequired[
        "capo_sagemaker.types.wait_interval_in_seconds.WaitIntervalInSeconds"
    ]
    """<p>The length of the baking period, during which SageMaker AI monitors alarms for each batch on the new fleet.</p>"""
    maximum_execution_timeout_in_seconds: NotRequired[
        "capo_sagemaker.types.maximum_execution_timeout_in_seconds.MaximumExecutionTimeoutInSeconds"
    ]
    """<p>The time limit for the total deployment. Exceeding this limit causes a timeout.</p>"""
    rollback_maximum_batch_size: NotRequired[
        "capo_sagemaker.types.inference_component_capacity_size.InferenceComponentCapacitySize"
    ]
    """<p>The batch size for a rollback to the old endpoint fleet. If this field is absent, the value is set to the default, which is 100% of the total capacity. When the default is used, SageMaker AI provisions the entire capacity of the old fleet at once during rollback.</p>"""

maximum_batch_size instance-attribute 

maximum_batch_size: NotRequired[InferenceComponentCapacitySize]

The batch size for each rolling step in the deployment process. For each step, SageMaker AI provisions capacity on the new endpoint fleet, routes traffic to that fleet, and terminates capacity on the old endpoint fleet. The value must be between 5% to 50% of the copy count of the inference component.

maximum_execution_timeout_in_seconds instance-attribute 

maximum_execution_timeout_in_seconds: NotRequired[MaximumExecutionTimeoutInSeconds]

The time limit for the total deployment. Exceeding this limit causes a timeout.

rollback_maximum_batch_size instance-attribute 

rollback_maximum_batch_size: NotRequired[InferenceComponentCapacitySize]

The batch size for a rollback to the old endpoint fleet. If this field is absent, the value is set to the default, which is 100% of the total capacity. When the default is used, SageMaker AI provisions the entire capacity of the old fleet at once during rollback.

wait_interval_in_seconds instance-attribute 

wait_interval_in_seconds: NotRequired[WaitIntervalInSeconds]

The length of the baking period, during which SageMaker AI monitors alarms for each batch on the new fleet.