Skip to content

create_bulk_import_job

create_bulk_import_job(
    job_name: BulkImportJobName,
    job_role_arn: ARN,
    files: Files,
    error_report_location: ErrorReportLocation,
    *,
    config_overrides: Optional[IoTSiteWiseClientConfig] = None,
    job_configuration: Optional[JobConfiguration] = None,
    adaptive_ingestion: Optional[AdaptiveIngestion] = None,
    delete_files_after_import: Optional[DeleteFilesAfterImport] = None,
    dataset_id: Optional[ID] = None,
    workspace_name: Optional[WorkspaceName] = None,
) -> CreateBulkImportJobResponse

Defines a job to ingest data to IoT SiteWise from Amazon S3. For more information, see Create a bulk import job (CLI) in the Amazon Simple Storage Service User Guide.

Before you create a bulk import job that ingests data into time series outside of a workspace, you must enable IoT SiteWise warm tier or IoT SiteWise cold tier. For more information about how to configure storage settings, see PutStorageConfiguration. This requirement doesn't apply to bulk import jobs that ingest data into a session dataset in a workspace (jobs that specify a workspaceName and datasetId). Those jobs don't use IoT SiteWise warm or cold tier storage.

Bulk import is designed to store historical data to IoT SiteWise.

  • Newly ingested data in the hot tier triggers notifications and computations.

  • After data moves from the hot tier to the warm or cold tier based on retention settings, it does not trigger computations or notifications.

  • Data older than 7 days does not trigger computations or notifications.

Parameters:

  • job_name (BulkImportJobName) –

    The unique name that helps identify the job request.

  • job_role_arn (ARN) –

    The ARN of the IAM role that allows IoT SiteWise to read Amazon S3 data.

  • files (Files) –

    The files in the specified Amazon S3 bucket that contain your data. You can specify up to 100 files for each bulk import job. Each file supports the following size limits:

    • Parquet files – Up to 256 MiB.

    • Other file formats – Up to 5 GiB.

  • error_report_location (ErrorReportLocation) –

    The Amazon S3 destination where errors associated with the job creation request are saved.

  • job_configuration (Optional[JobConfiguration], default: None ) –

    Contains the configuration information of a job, such as the file format used to save data in Amazon S3.

  • adaptive_ingestion (Optional[AdaptiveIngestion], default: None ) –

    If set to true, ingest new data into IoT SiteWise storage. Measurements with notifications, metrics and transforms are computed. If set to false, historical data is ingested into IoT SiteWise as is.

  • delete_files_after_import (Optional[DeleteFilesAfterImport], default: None ) –

    If set to true, your data files is deleted from S3, after ingestion into IoT SiteWise storage.

  • dataset_id (Optional[ID], default: None ) –

    The ID of the session dataset to ingest data into. Specify this field, together with workspaceName, to ingest data into a session dataset in a workspace.

  • workspace_name (Optional[WorkspaceName], default: None ) –

    The name of the workspace that contains the session dataset. Specify this field together with datasetId.

Raises:

  • ConflictingOperationException –

    Your request has conflicting operations. This can occur if you're trying to perform more than one operation on the same resource at the same time.

  • InternalFailureException –

    IoT SiteWise can't process your request right now. Try again later.

  • InvalidRequestException –

    The request isn't valid. This can occur if your request contains malformed JSON or unsupported characters. Check your request and try again.

  • LimitExceededException –

    You've reached the quota for a resource. For example, this can occur if you're trying to associate more than the allowed number of child assets or attempting to create more than the allowed number of properties for an asset model.

    For more information, see Quotas in the IoT SiteWise User Guide.

  • ResourceAlreadyExistsException –

    The resource already exists.

  • ResourceNotFoundException –

    The requested resource can't be found.

  • ThrottlingException –

    Your request exceeded a rate limit. For example, you might have exceeded the number of IoT SiteWise assets that can be created per second, the allowed number of messages per second, and so on.

    For more information, see Quotas in the IoT SiteWise User Guide.

  • UnknownServiceError –

    The service returned an error code this client does not model.

Source code in docs_src/capo_iotsitewise/_services/io_t_site_wise.py
def create_bulk_import_job(
    self,
    job_name: "capo_iotsitewise.types.bulk_import_job_name.BulkImportJobName",
    job_role_arn: "capo_iotsitewise.types.arn.ARN",
    files: "capo_iotsitewise.types.files.Files",
    error_report_location: "capo_iotsitewise.types.error_report_location.ErrorReportLocation",
    *,
    config_overrides: Optional[IoTSiteWiseClientConfig] = None,
    job_configuration: Optional[
        "capo_iotsitewise.types.job_configuration.JobConfiguration"
    ] = None,
    adaptive_ingestion: Optional[
        "capo_iotsitewise.types.adaptive_ingestion.AdaptiveIngestion"
    ] = None,
    delete_files_after_import: Optional[
        "capo_iotsitewise.types.delete_files_after_import.DeleteFilesAfterImport"
    ] = None,
    dataset_id: Optional["capo_iotsitewise.types.id.ID"] = None,
    workspace_name: Optional[
        "capo_iotsitewise.types.workspace_name.WorkspaceName"
    ] = None,
) -> "capo_iotsitewise.types.create_bulk_import_job_response.CreateBulkImportJobResponse":
    """<p>Defines a job to ingest data to IoT SiteWise from Amazon S3. For more information, see <a href="https://docs.aws.amazon.com/iot-sitewise/latest/userguide/CreateBulkImportJob.html">Create a bulk import job (CLI)</a> in the <i>Amazon Simple Storage Service User Guide</i>.</p> <important> <p>Before you create a bulk import job that ingests data into time series outside of a workspace, you must enable IoT SiteWise warm tier or IoT SiteWise cold tier. For more information about how to configure storage settings, see <a href="https://docs.aws.amazon.com/iot-sitewise/latest/APIReference/API_PutStorageConfiguration.html">PutStorageConfiguration</a>. This requirement doesn't apply to bulk import jobs that ingest data into a session dataset in a workspace (jobs that specify a <code>workspaceName</code> and <code>datasetId</code>). Those jobs don't use IoT SiteWise warm or cold tier storage.</p> <p>Bulk import is designed to store historical data to IoT SiteWise.</p> <ul> <li> <p>Newly ingested data in the hot tier triggers notifications and computations.</p> </li> <li> <p>After data moves from the hot tier to the warm or cold tier based on retention settings, it does not trigger computations or notifications.</p> </li> <li> <p>Data older than 7 days does not trigger computations or notifications.</p> </li> </ul> </important>

    Args:
        job_name: <p>The unique name that helps identify the job request.</p>
        job_role_arn: <p>The <a href="https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html">ARN</a> of the IAM role that allows IoT SiteWise to read Amazon S3 data.</p>
        files: <p>The files in the specified Amazon S3 bucket that contain your data. You can specify up to 100 files for each bulk import job. Each file supports the following size limits:</p> <ul> <li> <p>Parquet files – Up to 256 MiB.</p> </li> <li> <p>Other file formats – Up to 5 GiB.</p> </li> </ul>
        error_report_location: <p>The Amazon S3 destination where errors associated with the job creation request are saved.</p>
        job_configuration: <p>Contains the configuration information of a job, such as the file format used to save data in Amazon S3.</p>
        adaptive_ingestion: <p>If set to true, ingest new data into IoT SiteWise storage. Measurements with notifications, metrics and transforms are computed. If set to false, historical data is ingested into IoT SiteWise as is.</p>
        delete_files_after_import: <p>If set to true, your data files is deleted from S3, after ingestion into IoT SiteWise storage.</p>
        dataset_id: <p>The ID of the session dataset to ingest data into. Specify this field, together with <code>workspaceName</code>, to ingest data into a session dataset in a workspace.</p>
        workspace_name: <p>The name of the workspace that contains the session dataset. Specify this field together with <code>datasetId</code>.</p>

    Raises:
        capo_iotsitewise.errors.conflicting_operation_exception.ConflictingOperationException: <p>Your request has conflicting operations. This can occur if you're trying to perform more than one operation on the same resource at the same time.</p>
        capo_iotsitewise.errors.internal_failure_exception.InternalFailureException: <p>IoT SiteWise can't process your request right now. Try again later.</p>
        capo_iotsitewise.errors.invalid_request_exception.InvalidRequestException: <p>The request isn't valid. This can occur if your request contains malformed JSON or unsupported characters. Check your request and try again.</p>
        capo_iotsitewise.errors.limit_exceeded_exception.LimitExceededException: <p>You've reached the quota for a resource. For example, this can occur if you're trying to associate more than the allowed number of child assets or attempting to create more than the allowed number of properties for an asset model.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/iot-sitewise/latest/userguide/quotas.html">Quotas</a> in the <i>IoT SiteWise User Guide</i>.</p>
        capo_iotsitewise.errors.resource_already_exists_exception.ResourceAlreadyExistsException: <p>The resource already exists.</p>
        capo_iotsitewise.errors.resource_not_found_exception.ResourceNotFoundException: <p>The requested resource can't be found.</p>
        capo_iotsitewise.errors.throttling_exception.ThrottlingException: <p>Your request exceeded a rate limit. For example, you might have exceeded the number of IoT SiteWise assets that can be created per second, the allowed number of messages per second, and so on.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/iot-sitewise/latest/userguide/quotas.html">Quotas</a> in the <i>IoT SiteWise User Guide</i>.</p>
        capo_iotsitewise.errors.UnknownServiceError: The service returned an error code this client does not model.
    """

    def _handler(
        req: "OperationRequest[capo_iotsitewise.types.create_bulk_import_job_request.CreateBulkImportJobRequest]",
    ) -> OperationResponse[
        "capo_iotsitewise.types.create_bulk_import_job_response.CreateBulkImportJobResponse"
    ]:
        import capo_iotsitewise._operations.aws_io_t_site_wise.create_bulk_import_job

        output, http_response = (
            capo_iotsitewise._operations.aws_io_t_site_wise.create_bulk_import_job.create_bulk_import_job(
                req.options, req.input
            )
        )
        return OperationResponse(output=output, response=http_response)

    interceptors_, options_ = self.operation_options(
        config_overrides, OperationLogging("CreateBulkImportJob")
    )
    input_: capo_iotsitewise.types.create_bulk_import_job_request.CreateBulkImportJobRequest = {
        "job_name": job_name,
        "job_role_arn": job_role_arn,
        "files": files,
        "error_report_location": error_report_location,
    }
    if job_configuration is not None:
        input_["job_configuration"] = job_configuration
    if adaptive_ingestion is not None:
        input_["adaptive_ingestion"] = adaptive_ingestion
    if delete_files_after_import is not None:
        input_["delete_files_after_import"] = delete_files_after_import
    if dataset_id is not None:
        input_["dataset_id"] = dataset_id
    if workspace_name is not None:
        input_["workspace_name"] = workspace_name

    response = execute_pipeline(
        OperationRequest(input=input_, options=options_),
        handler=_handler,
        interceptors=list(interceptors_),
    )
    response.response.close()
    return response.output