Connecting to SaaS applications

You can land or replicate data from a large number of SaaS applications in Qlik Talend Data Integration.

To see all supported applications, go to Supported SaaS applications.

To rapidly develop connectors for the ever-growing number of cloud application sources, Qlik is creating applications connectors for Qlik Talend Data Integration using standard APIs and generative AI technology .

To accelerate the availability of new SaaS application connectors, we will initially develop some connectors for a specific use case and then releasing them with a Lite label, or marked with . For more information, see Classifications for SaaS application connectors.

Connecting to a SaaS application

Connecting to SaaS applications might require you to set up Data Movement gateway, depending on your use case. For more information, see When is Data Movement gateway required?.

1. The first step of connecting to a SaaS application is to add a connection to the data source. You can do this in several ways.

   • In Connections, click Create connection.

   • Click Create connection where source connections are listed (for example, in the data task setup wizard).

   Select Application in Category to filter the list of connectors to SaaS application connectors.

   You can also select from recently used connectors.

2. Select which application to connect to.

3. Complete the connection details to authenticate the connection. The connection details and authentication process are different for different SaaS applications.

   For more information, go to Connecting to SaaS applications and select your data source.

   Select Open connection metadata, and click Create when you are ready.

4. The metadata manager is displayed, where you can define metadata for the connection.

   Click Generate metadata to create metadata based on sampling the source data.

   For more information about metadata loading, see Scanning the data for metadata generation.

   Tip noteYou can also click Import metadata to import a metadata definition that is already generated. For more information, see Exporting and importing metadata.

5. Select the source datasets you want to use, and then click OK.

6. Select options for scanning the data. You can perform a full scan or a quick scan.

   A full scan will be more accurate, but may take a long time to process. If you select full scan, this data will be used in the initial load when landing data.

   You can also set default metadata settings.

   When you are ready, click Generate metadata.

When metadata generation is finished, you can use the connection in a landing or onboarding task.

Managing metadata

When you create the connection to the data source, you also need to define metadata for the datasets that are included. Metadata is used when generating schemas to normalize and define appropriate target tables. The metadata allows you to select which columns to be used in the replication to the target. When nested structures are returned from the source, the output tables are normalized to preserve the granularity of the data.

You can either:

• Generate metadata by sampling data from the source tables

• Import metadata that was exported from a connection with the same type of data source.

You can manage the metadata of a connection by clicking Metadata on the connection that you want to manage in Connections.

Scanning the data for metadata generation

When you are generating metadata, you can select to perform a full scan of the data, or a quick scan in Metadata loading.

• Full data scan will be more accurate, but may take a long time to process.

• Quick data scan is based on a sample of the data. This will be quicker, but not as accurate. You can select how many data samples to use in the scan with Number of data samples. Increase the number to improve accuracy.

  The number of records contained in a data sample depends on which data source is used.

You can also select how to set String columns size.

• Fixed

  Set a fixed string column size between 1 and 10000.

• Based on data values

  Set the string column size to the longest observed value for the field in the data sample, multiplied by the value in Multiplied by.

  If the field is empty for all rows in the sample, the column string size is set to the value in Default size when there is no value.

Selecting datasets

You can add datasets to the metadata.

• Click Select datasets.

  You can now select datasets to add to the metadata. If the dataset is already included in the metadata, it will be reloaded.

  Click OK when you are finished.

Deleting datasets

• To delete a dataset from the metadata, click ... on the dataset, and click Delete

Editing columns in a dataset

You can edit the columns in a dataset to set it as a key, make it nullable, or change data type.

1. Select the dataset.

2. Select the column to edit.

3. Click Edit.

Deleting columns in a dataset

1. Select the dataset.

2. Select the column to delete.

3. Click Delete.

Exporting and importing metadata

You can export metadata from a connection, and import it to another connection.

• Click Export metadata to export metadata.

• Click Import metadata to add metadata from an exported metadata file. The metadata file must have been exported from the same type of data source. If there are tables with the same name, you can either skip them when importing, or overwrite the existing metadata.

Reloading metadata

When there are structural changes in the source data, you must reload the metadata of the connection.

• To reload metadata for all datasets, click Reload metadata.

• To reload a specific dataset, click ... on the dataset, and click Reload metadata

For more information about metadata loading, see Scanning the data for metadata generation.