Skip to content

discover_input_schema

discover_input_schema(
    service_execution_role: RoleARN,
    *,
    config_overrides: Optional[KinesisAnalyticsV2ClientConfig] = None,
    resource_arn: Optional[ResourceARN] = None,
    input_starting_position_configuration: Optional[
        InputStartingPositionConfiguration
    ] = None,
    s3_configuration: Optional[S3Configuration] = None,
    input_processing_configuration: Optional[InputProcessingConfiguration] = None,
) -> DiscoverInputSchemaResponse

Infers a schema for a SQL-based Kinesis Data Analytics application by evaluating sample records on the specified streaming source (Kinesis data stream or Kinesis Data Firehose delivery stream) or Amazon S3 object. In the response, the operation returns the inferred schema and also the sample records that the operation used to infer the schema.

You can use the inferred schema when configuring a streaming source for your application. When you create an application using the Kinesis Data Analytics console, the console uses this operation to infer a schema and show it in the console user interface.

Parameters:

  • resource_arn (Optional[ResourceARN], default: None ) –

    The Amazon Resource Name (ARN) of the streaming source.

  • service_execution_role (RoleARN) –

    The ARN of the role that is used to access the streaming source.

  • input_starting_position_configuration (Optional[InputStartingPositionConfiguration], default: None ) –

    The point at which you want Kinesis Data Analytics to start reading records from the specified streaming source for discovery purposes.

  • s3_configuration (Optional[S3Configuration], default: None ) –

    Specify this parameter to discover a schema from data in an Amazon S3 object.

  • input_processing_configuration (Optional[InputProcessingConfiguration], default: None ) –

    The InputProcessingConfiguration to use to preprocess the records before discovering the schema of the records.

Raises:

Source code in docs_src/capo_kinesis_analytics_v2/_services/kinesis_analytics_v2.py
def discover_input_schema(
    self,
    service_execution_role: "capo_kinesis_analytics_v2.types.role_arn.RoleARN",
    *,
    config_overrides: Optional[KinesisAnalyticsV2ClientConfig] = None,
    resource_arn: Optional[
        "capo_kinesis_analytics_v2.types.resource_arn.ResourceARN"
    ] = None,
    input_starting_position_configuration: Optional[
        "capo_kinesis_analytics_v2.types.input_starting_position_configuration.InputStartingPositionConfiguration"
    ] = None,
    s3_configuration: Optional[
        "capo_kinesis_analytics_v2.types.s3_configuration.S3Configuration"
    ] = None,
    input_processing_configuration: Optional[
        "capo_kinesis_analytics_v2.types.input_processing_configuration.InputProcessingConfiguration"
    ] = None,
) -> "capo_kinesis_analytics_v2.types.discover_input_schema_response.DiscoverInputSchemaResponse":
    """<p>Infers a schema for a SQL-based Kinesis Data Analytics application by evaluating sample records on the specified streaming source (Kinesis data stream or Kinesis Data Firehose delivery stream) or Amazon S3 object. In the response, the operation returns the inferred schema and also the sample records that the operation used to infer the schema.</p> <p> You can use the inferred schema when configuring a streaming source for your application. When you create an application using the Kinesis Data Analytics console, the console uses this operation to infer a schema and show it in the console user interface. </p>

    Args:
        resource_arn: <p>The Amazon Resource Name (ARN) of the streaming source.</p>
        service_execution_role: <p>The ARN of the role that is used to access the streaming source.</p>
        input_starting_position_configuration: <p>The point at which you want Kinesis Data Analytics to start reading records from the specified streaming source for discovery purposes.</p>
        s3_configuration: <p>Specify this parameter to discover a schema from data in an Amazon S3 object.</p>
        input_processing_configuration: <p>The <a>InputProcessingConfiguration</a> to use to preprocess the records before discovering the schema of the records.</p>

    Raises:
        capo_kinesis_analytics_v2.errors.invalid_argument_exception.InvalidArgumentException: <p>The specified input parameter value is not valid.</p>
        capo_kinesis_analytics_v2.errors.invalid_request_exception.InvalidRequestException: <p>The request JSON is not valid for the operation.</p>
        capo_kinesis_analytics_v2.errors.resource_provisioned_throughput_exceeded_exception.ResourceProvisionedThroughputExceededException: <p>Discovery failed to get a record from the streaming source because of the Kinesis Streams <code>ProvisionedThroughputExceededException</code>. For more information, see <a href="http://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html">GetRecords</a> in the Amazon Kinesis Streams API Reference.</p>
        capo_kinesis_analytics_v2.errors.service_unavailable_exception.ServiceUnavailableException: <p>The service cannot complete the request.</p>
        capo_kinesis_analytics_v2.errors.unable_to_detect_schema_exception.UnableToDetectSchemaException: <p>The data format is not valid. Kinesis Data Analytics cannot detect the schema for the given streaming source.</p>
        capo_kinesis_analytics_v2.errors.unsupported_operation_exception.UnsupportedOperationException: <p>The request was rejected because a specified parameter is not supported or a specified resource is not valid for this operation. </p>
        capo_kinesis_analytics_v2.errors.UnknownServiceError: The service returned an error code this client does not model.
    """

    def _handler(
        req: "OperationRequest[capo_kinesis_analytics_v2.types.discover_input_schema_request.DiscoverInputSchemaRequest]",
    ) -> OperationResponse[
        "capo_kinesis_analytics_v2.types.discover_input_schema_response.DiscoverInputSchemaResponse"
    ]:
        import capo_kinesis_analytics_v2._operations.kinesis_analytics_20180523.discover_input_schema

        output, http_response = (
            capo_kinesis_analytics_v2._operations.kinesis_analytics_20180523.discover_input_schema.discover_input_schema(
                req.options, req.input
            )
        )
        return OperationResponse(output=output, response=http_response)

    interceptors_, options_ = self.operation_options(
        config_overrides, OperationLogging("DiscoverInputSchema")
    )
    input_: capo_kinesis_analytics_v2.types.discover_input_schema_request.DiscoverInputSchemaRequest = {
        "service_execution_role": service_execution_role
    }
    if resource_arn is not None:
        input_["resource_arn"] = resource_arn
    if input_starting_position_configuration is not None:
        input_["input_starting_position_configuration"] = (
            input_starting_position_configuration
        )
    if s3_configuration is not None:
        input_["s3_configuration"] = s3_configuration
    if input_processing_configuration is not None:
        input_["input_processing_configuration"] = input_processing_configuration

    response = execute_pipeline(
        OperationRequest(input=input_, options=options_),
        handler=_handler,
        interceptors=list(interceptors_),
    )
    response.response.close()
    return response.output