Skip to main content

Uploading on-premises data with Qlik DataTransfer

ON THIS PAGE

Uploading on-premises data with Qlik DataTransfer

Qlik DataTransfer uploads data from on-premises data sources to Qlik Sense SaaS tenants. Qlik DataTransfer can provide access to on-premise data sources that are otherwise unavailable to Qlik Sense SaaS. Qlik DataTransfer also can upload data from Qlik Sense Desktop and Qlik Sense Enterprise on Windows apps to Qlik Sense SaaS.

Qlik DataTransfer enables you to upload data from local data sources to Qlik Sense SaaS. You can upload data manually or schedule automatic data uploads to your tenant.

Qlik DataTransfer

Warning: Qlik DataTransfer should not be used to upload data from Qlik apps that use section access or have load scripts that control user access to data. Protected data might be exposed to unauthorized users.

You can upload data from the following kinds of data sources to spaces:

  • Data connections

    You can select tables and fields from ODBC and REST data sources that are available in your on-premises deployment.

  • Qlik apps

    You can transfer data from a Qlik Sense or QlikView app. The app can contain combined and transformed data from multiple data sources.

  • Data folders

    You can transfer data from a local folder or a network shared folder. If the folder is watched, and new or updated files are transferred automatically to the tenant.

Note:

You must have permission to add data sources in the destination space for the uploaded data.

You can select up to 10 dependent apps per data source for data connections and Qlik apps. Dependent apps are automatically reloaded every time data sources are uploaded to your tenant. Reloads in Qlik DataTransfer do not count against the reload limits in Qlik Sense SaaS tenants. Reloads of dependent apps in tenants do count against the reload limits.

Note: Using Qlik DataTransfer requires professional access and a developer role. For information on installing and configuring Qlik DataTransfer, see Installing Qlik DataTransfer .

Data connections

You can upload from on-premise data sources with Qlik connectors in Data connections. For example, you may have connections that are only accessible on-premises, such as ODBC DSN. With Qlik DataTransfer, you can connect to your data connection, load the data, and then upload the data as a QVD file to your tenant.

Note:

QVD files are temporarily stored in C:\ProgramData\Qlik\DataTransfer\DataUpload\qixdata. After uploading, they are deleted from this location.

Alternatively, the QVD file can be added to a local folder. This option is useful if you want to sync the QVD file with your cloud storage.

You manage your data connections in Data connections. You can add, edit, and delete your data connections. You can also manually upload datasets by selecting the data connection and clicking Upload. You can view details of the last upload by selecting a data connection and selecting Last load details.

Data connections can use new or existing connections. You can create new connections when you add connection data. You can also create and view connections in Connections details.

For more information on configuring specific connectors, see Qlik Connectors. For information on configuring ODBC DSN connections, see Loading data from ODBC data sources.

Note:

By default, the Qlik Connector for use with SAP NetWeaver is not available as a data connection in Qlik DataTransfer. You can add the connector to Qlik DataTransfer, however. For more information, see Adding Qlik Connector for use with SAP NetWeaver to Qlik DataTransfer.

Uploading data connections

  1. Click Data connections.
  2. Click Add.
  3. Enter a name for the connection data.
  4. Select the destination.
  5. Select the destination space or folder.
  6. Optionally, add a prefix.

    Prefixes are added to the QVD file name in the destination.

  7. Do one of the following:
    • To add connection data from an existing connection, select the connection.
    • To add a new connection, click Create connection, select a connection type, enter the connection parameters, and click Create.
  8. Select the tables and fields to load.
  9. Optionally, select the apps to reload whenever this data source is uploaded.
  10. Click Confirm.

Qlik apps

You can extract and upload data from Qlik Sense and QlikView app files (QVF and QVW). When you upload app data, Qlik DataTransfer runs the app data load script in the QVF or QVW. It then extracts all tables using binary load and stores them in a QVD file. The QVD is then uploaded to your Qlik Sense SaaS tenant.

Note:

QVD files are temporarily stored in C:\ProgramData\Qlik\DataTransfer\DataUpload\qixdata. After uploading, they are deleted from this location.

Alternatively, the QVD file can be added to a local folder. This option is useful if you want to sync the QVD file with your cloud storage.

App data is useful when you want to perform data transformations and loading procedures before data is added to the app. For example, it can be more efficient for app performance to perform an incremental load on your data in an app in Qlik Sense Desktop and then use Qlik DataTransfer to upload the resulting data to your tenant.

You can use app datasets to make data sources that are inaccessible to Qlik Sense SaaS accessible by loading them in a local Qlik Sense app. For example, Qlik Sense Desktop apps can connect to OLE DB data sources. Qlik DataTransfer can upload the OLE DB data from a desktop app and upload it to your tenant.

You manage your app data in Qlik apps. You can add, edit, and delete your app data. You can also manually upload datasets by selecting the app data and clicking Upload. You can view details of the last data upload by selecting an app dataset and selecting Last load details.

Qlik DataTransfer can only refresh data before uploading in QVF files authored in Qlik Sense Desktop. Qlik DataTransfer does not refresh the data in the app before uploading for QVF files authored in Qlik Sense Enterprise on Windows and downloaded from Qlik Sense SaaS. If you want data refreshed in a downloaded Qlik Sense app before upload, connections and data sources must be added to the app in Qlik Sense Desktop.

Uploading app data

Do the following:

  1. Click Qlik apps.
  2. Click Add.
  3. Enter a name for the app data.
  4. Select the destination.
  5. Select the destination space or folder.
  6. Optionally, add a prefix.

    Prefixes are added to the QVD file name in the destination.

  7. Click Next.
  8. Enter the file path to your Qlik app file and click Confirm.

    By default, Qlik Sense Desktop apps are stored in C:\Users\%USERPROFILE%\Documents\Qlik\Sense\Apps.

    Qlik Sense Enterprise on Windows apps are stored in the shared persistence storage folders configured during installation.

    By default, QlikView apps are stored in C:\ProgramData\QlikTech\Documents

  9. Optionally, select the apps to reload whenever this data source is uploaded.
  10. Click Confirm.

Data folders

You can select folders containing data files to upload to your tenant. Qlik DataTransfer watches selected folders. When the data in these files change or new files are added to the folder, Qlik DataTransfer automatically uploads the updated files to your tenant.

Note: If the connection to a network folder is disconnected, files changed in the folder during that period are not immediately uploaded when the connection is restored. These files are uploaded the next time a new file is added to that network folder, triggering an upload. Restarting the Qlik DataTransfer service will also trigger an upload of files that were changed during the period the connection was disconnected.

Qlik DataTransfer supports shared network folders, but not mapped network folders. For example, \\share\data would be supported. If \\share was mapped to Z:\, Z:\data would not be supported.

Note:

Data folders do not support scheduled uploads. Qlik DataTransfer will upload data when files are added or changed in the data folder.

Qlik DataTransfer watches and uploads files in the selected folder and its sub folders. When files are uploaded to tenants, they are prefixed with the folder names. For example, you are watching a folder named My Folder that has a sub folder named My SubFolder. This folder contains the file Sample.csv/ the uploaded file name in the tenant will be My Folder_My Subfolder_Sample.csv

You manage your data folders in Data folders. You can add folders, and edit or delete your selected folders. You can also manually upload data folder files by selecting the data folder and clicking Upload.

Uploading data folders

Do the following:

  1. In Qlik DataTransfer, select Data folders.
  2. Click Add.
  3. Enter a name.
  4. Enter the full path to the folder on the file system.
  5. Select the destination space.
  6. Optionally, select the apps to reload when this data source is uploaded.
  7. Optionally, add a prefix.

    Prefixes are added to the file names in the destination. By default, when files are uploaded to tenants, they are prefixed with the folder names.

  8. Click Confirm.

Scheduling uploads

You can schedule automatic uploads of your app and connection datasets on regular intervals. If you have selected any dependent apps for your data sources, they will automatically reload after the scheduled upload.

You can view details of the last upload by selecting a task and clicking Last execution details. You can view details of the last data upload by selecting an app dataset and selecting Last load details. Logs are available from C:\ProgramData\Qlik\DataTransfer\Log.

  1. In Qlik DataTransfer, select Scheduled uploads.
  2. Click Add.
  3. Enter a name.
  4. Select a data source to upload.
  5. Select the repeating interval and specify the date and time.
  6. Click Save.

Limitations

Qlik DataTransfer has the following limitations:

  • Qlik DataTransfer can upload files up to 6 GB in size.

  • You can upload a maximum of 200 tables or files per dataset. If your dataset has more than 200 tables or files, only the first 200 are uploaded. For example, if you select more than 200 tables in a data connection or Qlik app, or if you add a data folder with more than 200 files.
  • Qlik DataTransfer does not support connections to files in Essbase or Dropbox.
  • Files removed from a data folder must be manually removed from the spaces to which they were uploaded.
  • Qlik DataTransfer can only reload data in a QVF file if it has been created with Qlik Sense Desktop. If Qlik DataTransfer is installed on a server, you must copy your QVF files to your server. All data sources that are used in the QVF file must be accessible to the QVF load script when it is run on the server by Qlik DataTransfer.

  • QlikView apps are not automatically reloaded by Qlik DataTransfer when data sources in apps are updated with new data. QlikView apps must be manually reloaded and saved for Qlik DataTransfer to upload data to a SaaS edition of Qlik Sense tenant.

  • Qlik DataTransfer does not support the use of web proxies. If your system uses proxies, you must add a proxy bypass rule specifically for Qlik DataTransfer.