The data transformation service in Qlik Cloud Data Integration provides ELT capabilities to cloud data warehouses and data lakes. It is a key part of Qlik's vision to provide our customers with a raw to ready data pipeline.
The data transformation service is:
Cloud-based: Customers will design, deploy, and monitor data pipelines in Qlik Cloud with transformations executed via SQL in popular cloud targets.
Flexible, template-driven approach: Customers can define reusable transformations, set policies, and create custom design templates to simplify and accelerate the development of their data projects.
Automated: Automation of common methodologies and DataOps best practices allow customers to operationalize their transformation workloads quickly and reliably.
Integrated: Pipeline tasks are integrated with the data movement service to transform data in near real-time to analytics-ready data in your target cloud platform
The data transformation service contains the following functionality:
Creation of flexible, fit-for-purpose data pipelines
Rule-based, row-level transformations
Creation of new derived objects based on:
Source to target mappings
Custom SQL for more complex logic
Automated generation of star schema data marts
Define logical relationships between data sets
Choice to materialize data sets as tables or create as views
CDC (change data capture) support for low latency and incremental data movement
Push-down SQL execution to cloud DW platforms (Snowflake, Azure Synapse, Google BigQuery, Microsoft SQL Server)
Third-party data transformation in Qlik Cloud Data Integration refers to the process of registering already existing data that has been landed in the chosen cloud platform by external tools (including Qlik Replicate and Talend).
This means that customers can build workflows on top of and incorporate existing data into data pipelines without having to duplicate existing processes and consume the data twice. This includes to create transformation tasks, data cleansing, and data warehouse automation.
Use-cases for third party transformation include:
A temporary process during migration from legacy tools to Qlik Cloud Data Integration
Leveraging an existing cloud data warehouse or data lake for a new requirement
Allowing Qlik Cloud Data Integration to work with a propriety solution where connectivity is not available
Third party transformation supports key principles of Qlik: leave the data where it is, register it, understand it, make it qualitative, and start delivering it.
Did this page help you?
If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!