Working with QVD files

A QVD (QlikView Data) file is a file containing a table of data exported from Qlik Sense. QVD is a native Qlik format and can only be written to and read by Qlik Sense or QlikView. The file format is optimized for speed when reading data from a script but it is still very compact. Reading data from a QVD file is typically 10-100 times faster than reading from other data sources.

QVD files can be read in two modes: standard (fast) and optimized (faster). The selected mode is determined automatically by the script engine. Optimized mode can only be employed when all loaded fields are read without any transformations (formulas acting upon the fields), although the renaming of fields is allowed. A where clause causing Qlik Sense to unpack the records will also disable the optimized load.

A QVD file holds exactly one data table and consists of three parts:

1. Header.

   Information noteIf the QVD file was generated with QlikView the header is a well-formed XML header (in UTF-8 char set) describing the fields in the table, the layout of the subsequent information and other metadata.
2. Symbol tables in a byte-stuffed format.
3. Actual table data in a bit-stuffed format.

Purpose of QVD files

QVD files can be used for many purposes. At least four major uses can be easily identified. More than one may apply in any given situation:

Increasing load speed

By buffering non-changing or slowly-changing blocks of input data in QVD files, script execution becomes considerably faster for large data sets.

Decreasing load on database servers

The amount of data fetched from external data sources can also be greatly reduced. This reduces the workload on external databases and network traffic. Furthermore, when several scripts share the same data, it is only necessary to load it once from the source database into a QVD file. Other apps can make use of the same data through this QVD file.

Consolidating data from multiple apps

With the binary script statement, data can be loaded from a single app into another app, but with QVD files a script can combine data from any number of apps. This makes it possible for apps to consolidate similar data from different business units, for example.

Incremental load

In many common cases,, the QVD functionality can be used for incremental load by only loading new records from a growing database.

Creating QVD files

A QVD file can be created in two ways:

1. Explicit creation and naming using the store command in the script. State in the script that a previously-read table, or part thereof, is to be exported to an explicitly-named file at a location of your choice.
2. Automatic creation and maintenance from script. By preceding a LOAD or SELECT statement with the buffer prefix, Qlik Sense will automatically create a QVD file, which under certain conditions, can be used instead of the original data source when reloading data.

There is no difference between the resulting QVD files, with regard to reading speed, and so on.

Reading data from QVD files

A QVD file can be read into or accessed by the following methods:

1. Loading a QVD file as an explicit data source. QVD files can be referenced by a LOAD statement in the script, just like any other type of text files (csv, fix, dif, biff etc).

Example:  

LOAD * from xyz.qvd (qvd);

LOAD Name, RegNo from xyz.qvd (qvd);

LOAD Name as a, RegNo as b from xyz.qvd (qvd);

2. Automatic loading of buffered QVD files. When using the buffer prefix on LOAD or SELECT statements, no explicit statements for reading are necessary. Qlik Sense will determine the extent to which it will use data from the QVD file as opposed to acquiring data using the original LOAD or SELECT statement.
3. Accessing QVD files from the script. A number of script functions (all beginning with qvd) can be used for retrieving various information on the data found in the XML header of a QVD file.