In the panel that opens, select the type of connection you
want to create.
Example
Cosmos
DB
Select your engine
in the Engine list.
Information noteNote:
It is recommended to use the Remote Engine Gen2 rather than
the Cloud Engine for Design for advanced
processing of data.
If no Remote Engine Gen2 has been created from Talend Management Console or if it exists but appears as unavailable
which means it is not up and running, you will not be able to select
a Connection type in the list nor to
save the new connection.
The list of available connection types depends on the engine you
have selected.
Select the type of connection you want to create.
Here, select CosmosDB.
Fill in the connection properties to access your Azure Cosmos DB database as
described in Azure Cosmos DB properties, check the connection
and click Add dataset.
In the Add a new dataset
panel, name your dataset. In this example, the Cosmos DB collection will be used
to hold processed data about leads.
Example
leads
Fill in the required properties corresponding to the Cosmos DB
collection located in your Azure account.
Click Validate to save your dataset.
Do the same to add the Test connection and dataset that will
be used as source in your pipeline to populate the CRM.
In this example, a dataset named bank marketing
data with the following CSV schema is
used:
Click Add
pipeline on the Pipelines page. Your new pipeline opens.
Give the pipeline a meaningful name.
Example
Inserting bank marketing data
into a CosmosDB table
Click ADD SOURCE and
select your source dataset, bank marketing
data in the panel that opens.
Click to add processors to the pipeline, for example a Type
converter processor to convert data about balance from String type to
Double type.
Click to add a Filter processor to filter on leads that are
30 years old or older, that are managers and that have a balance superior to 2000
dollars.
Click the ADD
DESTINATION item on the pipeline to open the panel allowing to
select the Cosmos DB dataset in which your output data will be inserted.
In the Configuration
tab of the destination, click Main and:
Enable both the Create collection if not exists
and Auto ID generation options in order to create
a database collection and ID when executing the pipeline, in which the
data will be inserted.
Click Main and select
Insert in the Data action list to insert the data
into the existing dataset when executing the pipeline.
Click Save to
save your configuration.
On the top toolbar of Talend Cloud Pipeline Designer,
click the Run button to open the panel allowing you to select
your run profile.
Select your run profile in the list (for more information, see Run profiles), then click Run to
run your pipeline.
Results
Your pipeline is being executed, the data has been processed and filtered and
the output flow is inserted into the Azure Cosmos DB table you have defined.
You can check the log of your pipeline to see details about the volume of data sent to
Azure Cosmos DB.
Did this page help you?
If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!
to add processors to the pipeline, for example a Type
converter processor to convert data about balance from String type to
Double type.
