Skip to content

StepConfig

Source code in docs_src/capo_emr/types/step_config.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
class StepConfig(TypedDict, closed=True):
    name: NotRequired["capo_emr.types.xml_string_max_len256.XmlStringMaxLen256"]
    """<p>The name of the step.</p>"""
    action_on_failure: NotRequired["capo_emr.types.action_on_failure.ActionOnFailure"]
    """<p>The action to take when the step fails. Use one of the following values:</p> <ul> <li> <p> <code>TERMINATE_CLUSTER</code> - Shuts down the cluster.</p> </li> <li> <p> <code>CANCEL_AND_WAIT</code> - Cancels any pending steps and returns the cluster to the <code>WAITING</code> state.</p> </li> <li> <p> <code>CONTINUE</code> - Continues to the next step in the queue.</p> </li> <li> <p> <code>TERMINATE_JOB_FLOW</code> - Shuts down the cluster. <code>TERMINATE_JOB_FLOW</code> is provided for backward compatibility. We recommend using <code>TERMINATE_CLUSTER</code> instead.</p> </li> </ul> <p>If a cluster's <code>StepConcurrencyLevel</code> is greater than <code>1</code>, do not use <code>AddJobFlowSteps</code> to submit a step with this parameter set to <code>CANCEL_AND_WAIT</code> or <code>TERMINATE_CLUSTER</code>. The step is not submitted and the action fails with a message that the <code>ActionOnFailure</code> setting is not valid.</p> <p>If you change a cluster's <code>StepConcurrencyLevel</code> to be greater than 1 while a step is running, the <code>ActionOnFailure</code> parameter may not behave as you expect. In this case, for a step that fails with this parameter set to <code>CANCEL_AND_WAIT</code>, pending steps and the running step are not canceled; for a step that fails with this parameter set to <code>TERMINATE_CLUSTER</code>, the cluster does not terminate.</p>"""
    hadoop_jar_step: NotRequired[
        "capo_emr.types.hadoop_jar_step_config.HadoopJarStepConfig"
    ]
    """<p>The JAR file used for the step.</p>"""
    step_monitoring_configuration: NotRequired[
        "capo_emr.types.step_monitoring_configuration.StepMonitoringConfiguration"
    ]
    """<p>Object that holds configuration properties for logging.</p>"""

action_on_failure instance-attribute 

action_on_failure: NotRequired[ActionOnFailure]

The action to take when the step fails. Use one of the following values:

  • TERMINATE_CLUSTER - Shuts down the cluster.

  • CANCEL_AND_WAIT - Cancels any pending steps and returns the cluster to the WAITING state.

  • CONTINUE - Continues to the next step in the queue.

  • TERMINATE_JOB_FLOW - Shuts down the cluster. TERMINATE_JOB_FLOW is provided for backward compatibility. We recommend using TERMINATE_CLUSTER instead.

If a cluster's StepConcurrencyLevel is greater than 1, do not use AddJobFlowSteps to submit a step with this parameter set to CANCEL_AND_WAIT or TERMINATE_CLUSTER. The step is not submitted and the action fails with a message that the ActionOnFailure setting is not valid.

If you change a cluster's StepConcurrencyLevel to be greater than 1 while a step is running, the ActionOnFailure parameter may not behave as you expect. In this case, for a step that fails with this parameter set to CANCEL_AND_WAIT, pending steps and the running step are not canceled; for a step that fails with this parameter set to TERMINATE_CLUSTER, the cluster does not terminate.

hadoop_jar_step instance-attribute 

hadoop_jar_step: NotRequired[HadoopJarStepConfig]

The JAR file used for the step.

name instance-attribute 

name: NotRequired[XmlStringMaxLen256]

The name of the step.

step_monitoring_configuration instance-attribute 

step_monitoring_configuration: NotRequired[StepMonitoringConfiguration]

Object that holds configuration properties for logging.