Data transformation

The data transformation service in Qlik Cloud Data Integration provides ELT capabilities to cloud data warehouses and data lakes. It is a key part of Qlik's vision to provide our customers with a raw to ready data pipeline.

The data transformation service is:

• Cloud-based: Customers will design, deploy, and monitor data pipelines in Qlik Cloud with transformations executed via SQL in popular cloud targets.

• Flexible, template-driven approach: Customers can define reusable transformations, set policies, and create custom design templates to simplify and accelerate the development of their data projects.

• Automated: Automation of common methodologies and DataOps best practices allow customers to operationalize their transformation workloads quickly and reliably.

• Integrated: Pipeline tasks are integrated with the data movement service to transform data in near real-time to analytics-ready data in your target cloud platform

The data transformation service contains the following functionality:

• Creation of flexible, fit-for-purpose data pipelines

• Rule-based, row-level transformations

• Creation of new derived objects based on:

  • Source to target mappings

  • Custom SQL for more complex logic

  • Automated generation of star schema data marts

  • Define logical relationships between data sets

• Choice to materialize data sets as tables or create as views

• CDC (change data capture) support for low latency and incremental data movement

• Push-down SQL execution to cloud DW platforms (Snowflake, Azure Synapse, Google BigQuery, Microsoft SQL Server)

Third party data transformation

Third-party data transformation in Qlik Cloud Data Integration refers to the process of registering already existing data that has been landed in the chosen cloud platform by external tools (including Qlik Replicate and Talend).

This means that customers can build workflows on top of and incorporate existing data into data pipelines without having to duplicate existing processes and consume the data twice. This includes to create transformation tasks, data cleansing, and data warehouse automation.

Use-cases for third party transformation include:

• A temporary process during migration from legacy tools to Qlik Cloud Data Integration

• Leveraging an existing cloud data warehouse or data lake for a new requirement

• Allowing Qlik Cloud Data Integration to work with a propriety solution where connectivity is not available

Third party transformation supports key principles of Qlik: leave the data where it is, register it, understand it, make it qualitative, and start delivering it.