Skip to main content Skip to complementary content
Close announcements banner

Qlik Data Gateway - Data Movement

Qlik Data Gateway - Data Movement facilitates data movement from your enterprise data sources and SaaS applications to supported targets, over a strictly outbound, encrypted and mutually authenticated connection. By eliminating the need to open inbound firewall ports, Qlik Data Gateway - Data Movement provides a secure and trusted means of moving your enterprise data.

Protecting your data

A unique key-pair is established for each Data Movement gateway server connected to the Qlik Cloud tenant. This key pair is used for authentication and for end-to-end data encryption on top of the HTTPS transport level encryption. Moreover, as communication with Qlik Cloud and data transfer is initiated via outbound ports only, you do not need to open any inbound ports to your corporate network.

Qlik Cloud Government note

Qlik Cloud Government supports using Qlik Data Gateway - Data Movement when the Linux platform where Qlik Data Gateway - Data Movement is installed is configured to run in a FIPS 140-2 approved mode of operation, also known as "FIPS mode".

Information noteThe Data Movement gateway documentation always reflects the capabilities of the latest application version. If you’re not using the latest version, your system administrator is encouraged to upgrade immediately. For information on upgrading, see Upgrading and uninstalling Data Movement gateway

Common use cases

The following section highlights two common use cases: Moving data to a target that can only be accessed via a private connection, and moving data to targets that are directly accessible from Qlik Cloud.

Qlik Data Gateway - Data Movement plays a central role in the data movement process. The diagrams below illustrate how Data Movement gateway interacts with Qlik Cloud and your data sources to securely move data to supported targets.

Information note

The number of Data Movement gateways that need to be deployed depends on your specific environment and use case. If your data sources are in close proximity to each other and your target is directly accessible from Qlik Cloud, then a single Data Movement gateway will usually suffice. However, in the following cases, best practice is to use multiple gateways:

  • Ingesting data from data sources that are geographically distant from each other
  • Moving large volumes of data from multiple data sources
  • Moving data to a target that is inaccessible from Qlik Cloud (such as SQL Server in a VPC)

Supporting private connections in Qlik Cloud Data Integration

A private connection means that your data source or target is not exposed to the Internet, making it inaccessible from Qlik Cloud.

There are several types of private connection, examples of which include:

  • Snowflake AWS PrivateLink where public connectivity (via port 443) has been disabled
  • Microsoft SQL Server (or any other RDBMS) running in a VPC that has a private IP address only
Tip noteWhereas a Data Movement gateway must be deployed to access your data source, even if it is directly accessible from Qlik Cloud, deploying a Data Movement gateway to access your target platform is only required if the target is not directly accessible from Qlik Cloud.

The Data Movement gateway deployed to access your target performs ELT operations, which include some or all of the following: data registration, storage, transformations, and data marts.

Note that while it is possible to use the same gateway to access both the data source and the data target, using two gateways - one accessing the source and the other accessing the target - will often provide better performance. For example, if the source database is Oracle on-premises and the target database is Snowflake AWS PrivateLink, one gateway (defined in the Oracle source connection) will be used to land data from Oracle to Snowflake, while the other gateway (defined in the Snowflake target connection) will be used to perform the ELT operations. Using two gateways is especially recommended if the source and target databases are not in close proximity to each other. In such a scenario, the landing gateway should be as close as possible to the source, while the target gateway should be as close as possible to the target. If data is being ingested from multiple data sources, it might also make sense to set up a dedicated gateway for each data source, depending on both the type and the location of the data sources. An example of this type of setup is shown in the image below.

Moving data from distributed data sources to a target in a VPC and then performing ELT operations on the target

Moving data from enterprise data sources to supported targets

The high-level communication sequence is as follows:

  1. Data Movement gateway establishes a mutually authenticated and encrypted connection to Qlik Cloud, using HTTPS.

  2. Qlik Cloud sends task execution instructions to the Data Movement gateway, which in turn reports the task status to Qlik Cloud.

  3. Data Movement gateway retrieves the data from the data source and pushes it to the designated target.

Qlik Data Gateway - Data Movement - Moving data from enterprise data sources to supported targets

Moving data from SaaS applications to supported targets

The high-level communication sequence is as follows:

  1. Data Movement gateway establishes a mutually authenticated and encrypted connection to Qlik Cloud, using HTTPS.

  2. Qlik Cloud sends landing task execution instructions to the Data Movement gateway, which reports the task status back to Qlik Cloud.

  3. When the landing task is started, Qlik Cloud pulls the data from the SaaS application and stores it (encrypted with the tenant key).

  4. Data Movement gateway retrieves the SaaS application data from Qlik Cloud and pushes it to the designated target.

Tip noteIn this use case, Data Movement gateway does not have to be installed behind the firewall as neither the source (SaaS application) nor the target are located behind a firewall.

Qlik Data Gateway - Data Movement - Moving data from SaaS applications to supported targets

Moving data from SaaS applications and enterprise data sources to supported targets

Tip noteIn this use case, two Data Movement gateways are installed (on separate machines): one serving the enterprise data source and the other serving the SaaS application. The Data Movement gateway serving the SaaS application does not have to be installed behind the firewall as neither the source (SaaS application) nor the target are located behind a firewall.

The high-level communication sequence is a combination of the "Moving data from enterprise data sources" and "Moving data from SaaS applications" use cases described above.

Qlik Data Gateway - Data Movement - Moving data from SaaS applications and enterprise data sources to supported targets

Moving data from SaaS applications and enterprise data sources in a VPC to targets in the VPC

The high-level communication sequence for moving data from the SaaS application is as follows:

  1. Data Movement gateway establishes a mutually authenticated and encrypted connection to Qlik Cloud, using HTTPS.

  2. Qlik Cloud sends landing task execution instructions to the Data Movement gateway, which reports the task status back to Qlik Cloud.

  3. When the landing task is started, Qlik Cloud pulls the data from the SaaS application and stores it (encrypted with the tenant key).

  4. Data Movement gateway retrieves the SaaS application data from Qlik Cloud and pushes it to the designated target.

The high-level communication sequence for moving data from the enterprise data source is as follows:

  1. Data Movement gateway establishes a mutually authenticated and encrypted connection to Qlik Cloud, using HTTPS.

  2. Qlik Cloud sends landing task execution instructions to the Data Movement gateway, which reports the task status back to Qlik Cloud.

  3. Data Movement gateway retrieves the data from the data source and pushes it to the designated target.

Qlik Data Gateway - Data Movement - Moving data from SaaS applications and enterprise data sources in a VPC to targets in the VPC

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!