Loading and transforming data with scripting
Qlik Sense uses a data load script, which is managed in the Data load editor, to connect to and retrieve data from various data sources. A data source can be a data file, for example an Excel file or a .csv file. A data source can also be a database, for example a Google BigQuery or Salesforce database.
You can also load data using the Data manager, but when you want to create, edit and run a data load script you use the data load editor.
In the script, the fields and tables to load are specified. Scripting is often used to specify what data to load from your data sources. You can also manipulate the data structure by using script statements.
During the data load, Qlik Sense identifies common fields from different tables (key fields) to associate the data. The resulting data structure of the data in the app can be monitored in the data model viewer. Changes to the data structure can be achieved by renaming fields to obtain different associations between tables.
After the data has been loaded into Qlik Sense, it is stored in the app.
- To learn more about Data load editor, see Using the data load editor.
- To learn more about script syntax and data structures, see Understanding script syntax and data structures.
- To learn more about data and fields, see Guidelines for data and fields.
Analysis in Qlik Sense always happens while the app is not directly connected to its data sources. So, to refresh the data, you need to run the script to reload the data.
Interaction between Data manager and the data load script
When you add data tables in the Data manager, data load script code is generated. You can view the script code in the Auto-generated section of the data load editor. You can also choose to unlock and edit the generated script code, but if you do, the data tables will no longer be managed in the Data manager.
By default, data tables defined in the load script are not managed in Data manager. That is, you can see the tables in the data overview, but you cannot delete or edit the tables in Data manager, and association recommendations are not provided for tables loaded with the script. If you synchronize your scripted tables with Data manager, however, your scripted tables are added as managed scripted tables to Data manager.
If you have synchronized tables, you should not make changes in the data load editor with Data manager open in another tab.
To learn more about synchronized tables, see Synchronizing scripted tables in Data manager.
You can add script sections and develop code that enhances and interacts with the data model created in Data manager, but there are some areas where you need to be careful. The script code you write can interfere with the Data manager data model, and create problems in some cases, for example:
- Renaming or dropping tables added with Data manager in the script.
- Dropping fields from tables added with Data manager.
- Concatenation between tables added with Data manager and tables loaded in the script.
- Using the Qualify statement with fields in tables added with Data manager.
- Loading tables added with Data manager using Resident in the script.
- Adding script code after the generated code section. The resulting changes in the data model are not reflected in Data manager.
To learn more about the data load script, see Edit the data load script.