tCosmosDBSQLAPIOutput Standard properties

Property type

Either Built-in or Repository.

• Built-in: No property data is stored centrally.
• Repository: Select the repository file in which the properties are stored. The fields that follow are completed automatically using the data retrieved.

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

• Built-In: You create and store the schema locally for this component only.

• Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs.

• View schema: choose this option to view the schema only.

• Change to built-in property: choose this option to change the schema to Built-in for local changes.

• Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the Repository Content window.

URI

Enter the URI of your Cosmos user account. As the field prompts, the URI of a Cosmos DB user account is in the format of https://{your_account}.document.azure.com:443. You can obtain the URI on the Keys pane of Azure portal. Click Create an Azure Cosmos DB account for related information.

Primary key

Enter the primary key of your CosmosDB user account. You can obtain the pimary key on the Keys pane of Azure portal. Click Create an Azure Cosmos DB account for related information.

Database ID

Enter the the database name.

Collection ID

Enter the collection name.

Create collection if not exist

Select this option to have the collection specified in the Collection ID field created if it does not exist. This option is not available if Update or Delete is selected from the Data action drop-down list.

Data action

Select one of the following options.

• Insert: inserts documents.

• Update: replaces the existing documents with the incoming data but keeps the technical ID of these documents.

• Upsert: inserts a document if it does not exist otherwise it applies the same rules as Update.

• Delete: deletes documents.

Auto ID generation

Select this option to generate IDs for the documents processed.

tStatCatcher Statistics

Select this check box to collect the log data at the component level.

Connection mode

Specify the connection mode to be used. Two options are available: Gateway and DirectHttps. Click ConnectionMode for related information.

Max connection pool size

Set the maximum connection pool size. The default is 1000.

Consistency level

Set the consistency level. Click Consistency levels for related information.

Partition key

Enter the key of the partition whose documents will be removed. This option is available when Delete is selected from the Data action drop-down list.

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string.

NB_LINE

The number of rows processed. This is an After variable and it returns an integer.

Usage rule

tCosmosDBSQLAPIOutput executes the action defined on the collection in the database based on the flow incoming from the preceding component in the Job.