Managing data files
In Qlik Cloud Analytics, you can manage the data files in spaces from Space details > Data files. The management of data files includes renaming, duplicating, moving, deleting, and adding new data files.
This help topic focuses on the interface for managing data files in a space. To navigate to this interface, open a space in the catalog and click Space details > Data files.
Folder structures in spaces
Each space in Qlik Cloud Analytics contains its own structure of folders organized in a hierarchical fashion. Data files can be organized in a structure of nested folders. For example, within your personal space, you might want to have separate folders for Order Data and Budget Data.
The location of a file or folder within a space is shown in its path. The path uses / symbols to represent when a folder is nested within other folders.
For example, suppose that within your personal space, you have separate folders for Order Data and Budget Data. If you have a data file named Orders_2024_December.QVD in your Order Data folder, this file would be located in the following folder path within your personal space:
Personal/Order Data
Applicable space types
You can organize data files in a hierarchical folder structure in the following types of spaces:
-
Personal space
-
Shared space
-
Managed space
-
Data space
Spaces in Qlik Cloud Analytics
In Qlik Cloud Analytics, data files in personal, shared, and managed spaces can be organized in a hierarchical folder structure.
Spaces in Data Integration
Data spaces are commonly used to store output files from Data Integration projects. Data files in data spaces can be used for analytics. From Qlik Cloud Analytics and Qlik Talend Data Integration, you can organize data files in data spaces in hierarchical folder structures. It should be noted, however, that data files in data spaces should only be re-organized into nested folders for the purposes of use in analytics. If a data file that is generated from a data pipeline is relocated to a folder within the data space, the file will continue to be saved to the top-level path in the space (rather than any nested folder) whenever the pipeline is run again.
The behavior for a personal space within Qlik Talend Data Integration is the same as for data spaces.
Navigation
The Data files section of a space contains tools to help you organize and create file-based data content in the space.
Toolbar
The toolbar is located at the top of the interface above the path selector and table. In the toolbar, you can:
-
Search for data files in the space.
-
Filter files by owner.
-
Upload files.
-
Create folders.
-
Perform actions on multiple data files at a time.
-
Change the sorting of the files and folders in your view.
Change the sorting from the default Last modified option, and toggle between (ascending) and (descending) order.
-
Change the appearance of the files and folders in your view.
Select for the hierarchy view, which presents files and folders with indentation to visually display hierarchies. In hierarchy view, you need to select or expand a space to perform actions on specific files or folders.
Select for the list view, where files and folders are shown with a flat structure in the table along with a separate column displaying their specific paths.
Path selector
The path selector shows you what folder you currently have open. The breadcrumbs show you the folder hierarchy. Select a folder in the path selector to navigate forward or backward in relation to the current path. Select the empty area after the current path to see the folders contained within the current position, and to search for folders.
Table
The table shows all the files and folders in the space, or in the selected folder within the space. The table is where you manage the data files and folders. Select folders in the table to navigate to them in the path selector.
In the table, you can also view important information about items, such as their size, number of folders, time of last update, and ownership.
Creating folders in the space
Create a folder to organize content in any way you prefer.
Do the following:
-
In a space, click Space details and select Data files.
-
In the toolbar within the Data files window, click .
-
Define the name of the folder.
-
Under Path, you can choose where in the space's folder structure to add the new folder.
-
Click the path bar and select a containing folder from the drop down menu. Alternatively, type the path of a specific location where you want to add the folder.
-
Click Create.
Alternatively, add a folder by clicking next to an existing folder in the table and selecting Add folder. The new folder is added within the existing folder.
Folders are also automatically created in a space when they are referenced in Store statement. See Referencing space folder structure in app and script development.
Performing actions on multiple files and folders
You can use the options in the toolbar to perform the following actions on more than one file or folder at a time:
-
(Cut)
-
(Copy)
-
(Paste)
-
(Delete)
You can cut, copy, and paste items between spaces or to different locations within the same space. For example, you can cut multiple items and paste them into a different space.
Cut, copy, and paste actions can be used to move data files from one location to another. When changing file locations for data files referenced in load scripts, the load scripts need to be updated to reflect the new location. For more information, see Implications of moving data files used in load scripts.
Do the following:
- In a space, click Space details and select Data files.
-
In the table, select the items on which you want to perform an action.
Tip noteTo select more than one item at a time, click the first item, and then use a combination of keyboard commands and clicking to select more files. For example:
-
Hold Shift and click the last item you want to select. This selects all items from the first to last item.
-
Hold Ctrl (Windows) or command (macOS) and click additional items one at a time. This selects each additional item individually.
-
-
In the toolbar, click an action. The available actions are listed earlier in this section.
When cutting and pasting items between spaces, you need to open the new space and open Space details > Data files to perform the Paste action.
Adding a data file to a space
You can add data files to the space.
Data files can be up to 100 GB. However, when uploading very large data files (over 6 GB), you might experience constraints with engine capacity. These constraints are more likely to be encountered with QVD data files due to the memory usage necessary to load QVD files into the engine. For more information about increasing the capacity available, see Large app support.
Do the following:
- In a space, click Space details and select Data files.
- In the toolbar within the Data files window, click to upload a file.
-
Drag and drop your data files into the Add file dialog.
Alternatively, click Browse and navigate to your data files.
-
Under Path, you can choose where in the space's folder structure to add the file.
To add the file to a specific folder, click the path bar and select the folder from the drop down menu. Alternatively, type the path of the specific folder.
- Click Upload.
Alternatively, click next to a folder in the table and select Upload file to folder.
For additional ways to add data files, see Adding data files.
Performing cut, copy, and paste actions on a file or folder
Cut, copy, and paste actions can be used to move data files from one location to another. When changing file locations for data files referenced in load scripts, the load scripts need to be updated to reflect the new location. For more information, see Implications of moving data files used in load scripts.
Do the following:
-
In a space, click Space details and select Data files.
-
In the table, click next to a file or folder.
-
Click Cut, Copy, or Paste.
Tip noteAlternatively, you can select a file and use common keyboard shortcuts such as Ctrl+C (Windows) and command+C (macOS).
To perform these actions on more than one file or folder at a time, see Performing actions on multiple files and folders.
Renaming a data file or folder
Do the following:
-
In a space, click Space details and select Data files.
-
In the table, click next to a file or folder.
-
Click Rename.
-
Enter the new name for the file and click Save.
Any load scripts that reference the original file or folder name will need to be updated.
Duplicating a data file or folder
Do the following:
-
In a space, click Space details and select Data files.
-
In the table, click next to a file or folder.
-
Click Duplicate.
Moving a data file or folder
You can move files and folders. Files and folders can be moved to different spaces, or to different locations within the same space.
Do the following:
-
In a space, click Space details and select Data files.
-
In the table, click next to a file or folder.
-
Click Move.
-
Under Path, select a space. This can be the current space or a new space.
-
Reselect the path bar, or click , to define a specific folder within the selected space.
Alternatively, type the path to define a specific folder.
-
To open to the new location after moving the data file, select Navigate to path.
-
Click Move.
You can also move multiple items at a time. To do this, use the Cut and Paste actions. See Performing actions on multiple files and folders.
Implications of moving data files used in load scripts
If you move a data file to a new location within a space, any load scripts that reference that data file need to be updated to reference the new file path. The same consideration applies for files that are moved from one space to another.
For example, suppose the load script of your app or script references the following file in a personal space:
Now, suppose that you move the data files to the space TeamSharedSpace and move orders.csv into a new folder in the space, named Sales Analysis. All apps and scripts that load data from orders.csv need to have the file reference updated to the following in the load script:
Otherwise, reloads will fail.
For more information about how to refer to space folder locations in load scripts, see Referencing space folder structure in app and script development.
Deleting a data file or folder
Do the following:
-
In a space, click Space details and select Data files.
-
In the table, click next to a file or folder.
-
Click Delete.
Tip noteAlternatively, you can select a file and enter Delete on your keyboard. - In the dialog, click Delete to confirm the deletion.
You can also delete multiple items at a time. See Performing actions on multiple files and folders.
Performing other actions on a data file
In the Data files interface, you can also navigate directly to other tools for performing analysis of a specific file. The following options are available when you use the menu on a file:
-
Impact analysis: Investigate the potential effects of making changes to data in the dataset.
-
Lineage: Understand more about the history of the dataset.
-
Open dataset: Open the dataset, where you can access an overview and profile information for the file.
Space permissions
Data files are a type of data source in Qlik Cloud Analytics.
Shared space members with any of the following roles can manage data sources in that space:
-
Owner
-
Can manage
-
Can edit
-
Can edit data in apps
Managed space members with any of the following roles can manage data sources in that space:
-
Owner
-
Can manage
-
Can operate (with Professional or Full User entitlement)
Space members with the Can consume data role can view data sources in the space, and they can consume the data sources where they have permission to create apps.
For more information about the space role requirements for data source actions, see:
-
Shared space permissions for users with Professional or Full User entitlement
-
Shared space permissions for users with Analyzer entitlement
-
Managed space permissions for users with Professional or Full User entitlement
-
Managed space permissions for users with Analyzer entitlement
Referencing space folder structure in app and script development
Rules for valid space folder paths
In Qlik Cloud Analytics, the syntax for referring to a file in the load script is:
[lib:// (space_name):(connection_name)/(folder_name)/(file_name)]
The full folder path must be provided. Relative paths are not supported.
If the space does not exist, the statement cannot be executed and returns an error.
The folder path can include multiple folders in a hierarchical structure. Each folder_name in the path must be delimited with / characters.
The following are not allowed in the path when referencing files residing in a folder structure:
-
//
-
\
-
\\
-
/./
-
/../
Trailing period symbols (.) in the space, folder, or file name are acceptable as long as they do not appear in the configurations in the above list.
Examples: Non-compliant syntax
These examples are considered invalid syntax for the path:
Depending on the type of script statement being used, non-compliant syntax might cause reload failure, or could still be executed with different behavior. For specific details, see:
Loading data from files
In Qlik Cloud Analytics, when loading data from a data file within a load script, you can use a LOAD statement or insert QVS code. In both cases, hierarchical space folder structures can be referenced in your script statement. Follow the rules for valid space folder paths. Additionally, the file and all referenced folders need to already exist in the specified location. Otherwise, the reload fails.
LOAD statements in the load script
Examples: Reference to file at the top-level path of a space
This example loads the file orders.csv from a personal space. The file is not located within any specific folder, but instead is located at the top-level path in the space.
LOAD * FROM [lib://DataFiles/orders.csv];
This example loads the file orders.csv from a shared, managed, or data space. The name of the space is TeamSharedSpace. The file is not located within any specific folder, but instead is located at the top-level path in the space.
LOAD * FROM [lib://TeamSharedSpace:DataFiles/orders.csv];
Examples: References to specific folder path within a space
This example loads the file orders_Europe.csv from the folder Orders By Region within a personal space.
LOAD * FROM [lib://DataFiles/Orders By Region/orders_Europe.csv];
This example loads the file orders_Europe.csv from the folder Orders By Region within a shared, managed, or data space. The name of the space is SalesSpace.
LOAD * FROM [lib://SalesSpace:DataFiles/Orders By Region/orders_Europe.csv];
Inserting QVS code
Example: Referencing QVS file at the top-level path of a space
In this example, the file test.qvs is referenced in the load script. The file is located in the space TeamSharedSpace. The file is not located within any specific folder, but instead is located at the top-level path in the space.
$(Must_Include=lib://TeamSharedSpace:DataFiles/test.qvs)
Example: Referencing QVS file in a specific folder path within a space
In this example, the file test.qvs is referenced in the load script. The file is located in the folder SalesData within the space TeamSharedSpace.
$(Must_Include=lib://TeamSharedSpace:DataFiles/SalesData/test.qvs)
Storing data into files
When you use a Store statement in a load script in Qlik Cloud, you can reference the exact folder in a space where you want to store the created files. Follow the rules for valid space folder paths. Additionally, the following behaviors also apply and are unique to Store statements:
-
If the space exists but the specified folder path does not, the path will be created in the space. One or more folders might be created depending on the defined folder structure. The file is generated and stored there.
-
If the space exists but the specified folder path uses invalid folder delimiters (for example, \, \\, //, /./, or /../), the Store statement is still executed. The output file is instead saved at the top-level path in the target space and will have a name that contains its full path, not only the specified name.
-
If the space does not exist, the Store statement cannot be executed and returns an error.
The following examples demonstrate how to refer to a specific folder path in the space within your Store statement.
Example: Storing file at the top-level path of a space
In this example, the entire table mytable is exported into Qlik Cloud Analytics as a file named myfile.qvd. The file is added to the shared space TeamSharedSpace. The file is not located within any specific folder, but instead is located at the top-level path in the space.
Store * from mytable into [lib://TeamSharedSpace:DataFiles/myfile.qvd];
Example: Storing file to a specific folder path within a space
In this example, the entire table mytable is exported into Qlik Cloud Analytics as a file named myfile.qvd. The file is added to the folder SalesData within the space TeamSharedSpace.
Store * from mytable into [lib://TeamSharedSpace:DataFiles/SalesData/myfile.qvd];
Administering data files
Tenant administrators and analytics administrators can manage all data files in the space from the Administration activity center. For more information, see Managing data files for tenant and analytics administrators.