Skip to content

SparkSQL

Source code in docs_src/capo_glue/types/spark_sql.py 
17
18
19
20
21
22
23
24
25
26
27
class SparkSQL(TypedDict, closed=True):
    name: "capo_glue.types.node_name.NodeName"
    """<p>The name of the transform node.</p>"""
    inputs: "capo_glue.types.many_inputs.ManyInputs"
    """<p>The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.</p>"""
    sql_query: "capo_glue.types.sql_query.SqlQuery"
    """<p>A SQL query that must use Spark SQL syntax and return a single data set.</p>"""
    sql_aliases: "capo_glue.types.sql_aliases.SqlAliases"
    """<p>A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify <code>From</code> as MyDataSource, and <code>Alias</code> as SqlName, then in your SQL you can do:</p> <p> <code>select * from SqlName</code> </p> <p>and that gets data from MyDataSource.</p>"""
    output_schemas: NotRequired["capo_glue.types.glue_schemas.GlueSchemas"]
    """<p>Specifies the data schema for the SparkSQL transform.</p>"""

inputs instance-attribute 

inputs: ManyInputs

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

name instance-attribute 

name: NodeName

The name of the transform node.

output_schemas instance-attribute 

output_schemas: NotRequired[GlueSchemas]

Specifies the data schema for the SparkSQL transform.

sql_aliases instance-attribute 

sql_aliases: SqlAliases

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

sql_query instance-attribute 

sql_query: SqlQuery

A SQL query that must use Spark SQL syntax and return a single data set.