Working with QVD files

A QVD (QlikView Data) file is a file containing a table of data exported from Qlik Sense. QVD is a native Qlik format and can only be written to and read by Qlik Sense or QlikView. The file format is optimized for speed when reading data from a script but it is still very compact. Reading data from a QVD file is typically 10-100 times faster than reading from other data sources.

QVD files can be read in two modes: standard (fast) and optimized (faster). The selected mode is determined automatically by the script engine.

There are some limitations regarding optimized loads. It is possible to rename fields, but any of the operations mentioned here will disable the optimized load and result in a standard load.

• Any transformations on the fields that are loaded.
• Using a where clause causing Qlik Sense to unpack the records.
• Using Map on a field that is loaded.

Purpose of QVD files

QVD files can be used for many purposes and more than one may apply in any given situation. At least four major uses can be easily identified:

• Increasing load speed

By buffering non-changing or slowly-changing blocks of input data in QVD files, script execution becomes considerably faster for large data sets.

• Decreasing load on database servers

The amount of data fetched from external data sources can also be greatly reduced. This reduces the workload on external databases and network traffic. Furthermore, when several scripts share the same data, it is only necessary to load it once from the source database into a QVD file. Other apps can make use of the same data through this QVD file.

• Consolidating data from multiple apps

With the binary script statement, data can be loaded from a single app into another app, but with QVD files a script can combine data from any number of apps. This makes it possible for apps to consolidate similar data from different business units, for example.

• Incremental

In many common cases, the QVD functionality can be used for incremental load by loading only new records from a growing database.

For more information, see Loading new and updated records with incremental load.

Creating QVD files

A QVD file can be created in two ways:

• Explicit creation and naming using the store command in the script. State in the script that a previously-read table, or part thereof, is to be exported to an explicitly-named file at a location of your choice.

  For more information, see Store.

• Automatic creation and maintenance from script. When you precede a LOAD or SELECT statement with the buffer prefix, Qlik Sense will automatically create a QVD file, which, under certain conditions, can be used instead of the original data source when reloading data.

  For more information, see Buffer.

There is no difference between the resulting QVD files with regard to reading speed.

Reading data from QVD files

A QVD file can be read or accessed by the following methods:

• Loading a QVD file as an explicit data source. QVD files can be referenced by a LOAD statement in the script, just like any other type of text files (csv, fix, dif, biff etc).

Example:  

LOAD * from xyz.qvd (qvd);

LOAD Name, RegNo from xyz.qvd (qvd);

LOAD Name as a, RegNo as b from xyz.qvd (qvd);

• Automatic loading of buffered QVD files. When you use the buffer prefix on LOAD or SELECT statements, no explicit statements for reading are necessary. Qlik Sense will determine the extent to which it will use data from the QVD file as opposed to acquiring data using the original LOAD or SELECT statement.
• Accessing QVD files from the script. A number of script functions (all beginning with qvd) can be used for retrieving various information on the data found in the XML header of a QVD file.

QVD format

A QVD file holds exactly one data table and consists of three parts:

• Header.

  Information noteIf the QVD file was generated with QlikView the header is a well-formed XML header (in UTF-8 char set) describing the fields in the table, the layout of the subsequent information and other metadata.
• Symbol tables in a byte-stuffed format.
• Actual table data in a bit-stuffed format.