Skip to content

DevEndpoint

Source code in docs_src/capo_glue/types/dev_endpoint.py
class DevEndpoint(TypedDict, closed=True):
    endpoint_name: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The name of the <code>DevEndpoint</code>.</p>"""
    role_arn: NotRequired["capo_glue.types.role_arn.RoleArn"]
    """<p>The Amazon Resource Name (ARN) of the IAM role used in this <code>DevEndpoint</code>.</p>"""
    security_group_ids: NotRequired["capo_glue.types.string_list.StringList"]
    """<p>A list of security group identifiers used in this <code>DevEndpoint</code>.</p>"""
    subnet_id: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The subnet ID for this <code>DevEndpoint</code>.</p>"""
    yarn_endpoint_address: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The YARN endpoint address used by this <code>DevEndpoint</code>.</p>"""
    private_address: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>A private IP address to access the <code>DevEndpoint</code> within a VPC if the <code>DevEndpoint</code> is created within one. The <code>PrivateAddress</code> field is present only when you create the <code>DevEndpoint</code> within your VPC.</p>"""
    zeppelin_remote_spark_interpreter_port: "capo_glue.types.integer_value.IntegerValue"
    """<p>The Apache Zeppelin port for the remote Apache Spark interpreter.</p>"""
    public_address: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The public IP address used by this <code>DevEndpoint</code>. The <code>PublicAddress</code> field is present only when you create a non-virtual private cloud (VPC) <code>DevEndpoint</code>.</p>"""
    status: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The current status of this <code>DevEndpoint</code>.</p>"""
    worker_type: NotRequired["capo_glue.types.worker_type.WorkerType"]
    """<p>The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.</p> <ul> <li> <p>For the <code>Standard</code> worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.</p> </li> <li> <p>For the <code>G.1X</code> worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.</p> </li> <li> <p>For the <code>G.2X</code> worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.</p> </li> </ul> <p>Known issue: when a development endpoint is created with the <code>G.2X</code> <code>WorkerType</code> configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk. </p>"""
    glue_version: NotRequired["capo_glue.types.glue_version_string.GlueVersionString"]
    """<p>Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints. </p> <p>For more information about the available Glue versions and corresponding Spark and Python versions, see <a href="https://docs.aws.amazon.com/glue/latest/dg/add-job.html">Glue version</a> in the developer guide.</p> <p>Development endpoints that are created without specifying a Glue version default to Glue 0.9.</p> <p>You can specify a version of Python support for development endpoints by using the <code>Arguments</code> parameter in the <code>CreateDevEndpoint</code> or <code>UpdateDevEndpoint</code> APIs. If no arguments are provided, the version defaults to Python 2.</p>"""
    number_of_workers: NotRequired["capo_glue.types.nullable_integer.NullableInteger"]
    """<p>The number of workers of a defined <code>workerType</code> that are allocated to the development endpoint.</p> <p>The maximum number of workers you can define are 299 for <code>G.1X</code>, and 149 for <code>G.2X</code>. </p>"""
    number_of_nodes: "capo_glue.types.integer_value.IntegerValue"
    """<p>The number of Glue Data Processing Units (DPUs) allocated to this <code>DevEndpoint</code>.</p>"""
    availability_zone: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The Amazon Web Services Availability Zone where this <code>DevEndpoint</code> is located.</p>"""
    vpc_id: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The ID of the virtual private cloud (VPC) used by this <code>DevEndpoint</code>.</p>"""
    extra_python_libs_s3_path: NotRequired[
        "capo_glue.types.generic_string.GenericString"
    ]
    """<p>The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your <code>DevEndpoint</code>. Multiple values must be complete paths separated by a comma.</p> <note> <p>You can only use pure Python libraries with a <code>DevEndpoint</code>. Libraries that rely on C extensions, such as the <a href="http://pandas.pydata.org/">pandas</a> Python data analysis library, are not currently supported.</p> </note>"""
    extra_jars_s3_path: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The path to one or more Java <code>.jar</code> files in an S3 bucket that should be loaded in your <code>DevEndpoint</code>.</p> <note> <p>You can only use pure Java/Scala libraries with a <code>DevEndpoint</code>.</p> </note>"""
    failure_reason: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The reason for a current failure in this <code>DevEndpoint</code>.</p>"""
    last_update_status: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The status of the last update.</p>"""
    created_timestamp: NotRequired["capo_glue.types.timestamp_value.TimestampValue"]
    """<p>The point in time at which this DevEndpoint was created.</p>"""
    last_modified_timestamp: NotRequired[
        "capo_glue.types.timestamp_value.TimestampValue"
    ]
    """<p>The point in time at which this <code>DevEndpoint</code> was last modified.</p>"""
    public_key: NotRequired["capo_glue.types.generic_string.GenericString"]
    """<p>The public key to be used by this <code>DevEndpoint</code> for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.</p>"""
    public_keys: NotRequired["capo_glue.types.public_keys_list.PublicKeysList"]
    """<p>A list of public keys to be used by the <code>DevEndpoints</code> for authentication. Using this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.</p> <note> <p>If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the <code>UpdateDevEndpoint</code> API operation with the public key content in the <code>deletePublicKeys</code> attribute, and the list of new keys in the <code>addPublicKeys</code> attribute.</p> </note>"""
    security_configuration: NotRequired["capo_glue.types.name_string.NameString"]
    """<p>The name of the <code>SecurityConfiguration</code> structure to be used with this <code>DevEndpoint</code>.</p>"""
    arguments: NotRequired["capo_glue.types.map_value.MapValue"]
    """<p>A map of arguments used to configure the <code>DevEndpoint</code>.</p> <p>Valid arguments are:</p> <ul> <li> <p> <code>"--enable-glue-datacatalog": ""</code> </p> </li> </ul> <p>You can specify a version of Python support for development endpoints by using the <code>Arguments</code> parameter in the <code>CreateDevEndpoint</code> or <code>UpdateDevEndpoint</code> APIs. If no arguments are provided, the version defaults to Python 2.</p>"""

arguments instance-attribute

arguments: NotRequired[MapValue]

A map of arguments used to configure the DevEndpoint.

Valid arguments are:

  • "--enable-glue-datacatalog": ""

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

availability_zone instance-attribute

availability_zone: NotRequired[GenericString]

The Amazon Web Services Availability Zone where this DevEndpoint is located.

created_timestamp instance-attribute

created_timestamp: NotRequired[TimestampValue]

The point in time at which this DevEndpoint was created.

endpoint_name instance-attribute

endpoint_name: NotRequired[GenericString]

The name of the DevEndpoint.

extra_jars_s3_path instance-attribute

extra_jars_s3_path: NotRequired[GenericString]

The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint.

You can only use pure Java/Scala libraries with a DevEndpoint.

extra_python_libs_s3_path instance-attribute

extra_python_libs_s3_path: NotRequired[GenericString]

The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

You can only use pure Python libraries with a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not currently supported.

failure_reason instance-attribute

failure_reason: NotRequired[GenericString]

The reason for a current failure in this DevEndpoint.

glue_version instance-attribute

glue_version: NotRequired[GlueVersionString]

Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

Development endpoints that are created without specifying a Glue version default to Glue 0.9.

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

last_modified_timestamp instance-attribute

last_modified_timestamp: NotRequired[TimestampValue]

The point in time at which this DevEndpoint was last modified.

last_update_status instance-attribute

last_update_status: NotRequired[GenericString]

The status of the last update.

number_of_nodes instance-attribute

number_of_nodes: IntegerValue

The number of Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

number_of_workers instance-attribute

number_of_workers: NotRequired[NullableInteger]

The number of workers of a defined workerType that are allocated to the development endpoint.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

private_address instance-attribute

private_address: NotRequired[GenericString]

A private IP address to access the DevEndpoint within a VPC if the DevEndpoint is created within one. The PrivateAddress field is present only when you create the DevEndpoint within your VPC.

public_address instance-attribute

public_address: NotRequired[GenericString]

The public IP address used by this DevEndpoint. The PublicAddress field is present only when you create a non-virtual private cloud (VPC) DevEndpoint.

public_key instance-attribute

public_key: NotRequired[GenericString]

The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

public_keys instance-attribute

public_keys: NotRequired[PublicKeysList]

A list of public keys to be used by the DevEndpoints for authentication. Using this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API operation with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

role_arn instance-attribute

role_arn: NotRequired[RoleArn]

The Amazon Resource Name (ARN) of the IAM role used in this DevEndpoint.

security_configuration instance-attribute

security_configuration: NotRequired[NameString]

The name of the SecurityConfiguration structure to be used with this DevEndpoint.

security_group_ids instance-attribute

security_group_ids: NotRequired[StringList]

A list of security group identifiers used in this DevEndpoint.

status instance-attribute

status: NotRequired[GenericString]

The current status of this DevEndpoint.

subnet_id instance-attribute

subnet_id: NotRequired[GenericString]

The subnet ID for this DevEndpoint.

vpc_id instance-attribute

vpc_id: NotRequired[GenericString]

The ID of the virtual private cloud (VPC) used by this DevEndpoint.

worker_type instance-attribute

worker_type: NotRequired[WorkerType]

The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

  • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

  • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

  • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

yarn_endpoint_address instance-attribute

yarn_endpoint_address: NotRequired[GenericString]

The YARN endpoint address used by this DevEndpoint.

zeppelin_remote_spark_interpreter_port instance-attribute

zeppelin_remote_spark_interpreter_port: IntegerValue

The Apache Zeppelin port for the remote Apache Spark interpreter.