Google Cloud Storage
The Qlik Google Cloud Storage Web Storage Provider Connector lets you fetch your stored data from Google Cloud Storage buckets, allowing you to stream data directly into your Qlik Sense app from your Google Cloud Storage account, just as you would from a local file. You can also export and upload compiled table data into your remote Google Cloud Storage.
Supported offerings
- Qlik Sense Business
- Qlik Sense Enterprise SaaS (as of v0.93)
- Qlik Sense Enterprise on Windows (as of v4.18.0 / May 2022)
- Qlik Sense Desktop (as of v4.18.0 / May 2022)
Connecting to an Google Cloud Storage account
You need to select the connector in Qlik Sense, and authenticate the connector with your Google Cloud Storage credentials.
For example: a Qlik Sense deployment in the North American region may have significantly slower download and upload speeds when connecting to a Google Cloud Storage bucket in the Asia-Pacific region.
You can access the connector through Data manager and the Data load editor.
- Data manager: Click + and then Add new connection. Select Google Cloud Storage under Connect to new data source.
- Data load editor: Click Create new connection and select Google Cloud Storage from the list.
Authentication
You need to authenticate the connector with your Google Cloud Storage credentials:
- Service account key file: You need one or more service accounts with appropriate roles and permissions in the IAM settings in the Google Cloud console. The simplest option is a service account with a storage admin role and no conditions. You will need to create a JSON key for each service account you wish to use. This key file should be downloaded and stored securely. Note that P12 keys are not supported, as they are considered legacy. In order to authenticate, you will need the key file and the bucket name you wish to connect to. These details are stored securely in the connection.
- Bucket name: You can find bucket names in the Google Cloud console storage page.
- Name: In Data load editor, you can specify a connection name or leave the default name for the connection. You cannot rename connections in Data manager.
In the Data load editor, you can verify the connection before you create it by clicking Test Connection.
When you are finished setting up your data connection, click Create. The connection credentials are verified automatically.
Connector location
The location of the connection depends on how you select data.
-
Data manager: It will be saved under File Locations.
-
Data load editor: It will be saved under Data connections.
Selecting data
When you create a Google Cloud Storage connection with Data manager, the data selection step follows immediately. When you create a data connection with the Data load editor, you need to open the data connection before you can select data.
Do the following:
-
Navigate to the file path location of your Google Cloud Storage account.
-
Locate your files by clicking on the folder names.
You can navigate backwards by clicking on the previous folder in the path field or by clicking
. You can filter the file types that are shown by using the File type drop-down menu.
-
Select a file. Then, select the data tables and the data fields that you want to load.
You can rename data fields before loading them into Qlik Sense by clicking on the field name and entering a new name. The new name is mapped by Qlik Sense as an alias to that field name in the database.
Data file properties
If you are loading data from an Excel file or from a delimited file, you may need to adjust the data file properties. The following table describes the data file properties for these file types:
Parameter | Option | Description | For file type |
---|---|---|---|
Field names | Embedded field names | Set to specify if the table contains Embedded field names or No field names. Typically, in an Excel spreadsheet, the first row contains the embedded field names. |
Delimited files .xls/.xlsx |
Field names | No field names | Select if the fields are not labeled. If you select this option, fields will be named A,B,C... |
Delimited files .xls/.xlsx |
Header size | - | Set to the number of header rows to ignore. Typically, you want to exclude rows that contain general information that is not in a columnar format. |
Delimited files .xls/.xlsx |
Delimiter | - | Set the field delimiter. Select a standard delimiter (comma, semicolon, space, tab) or a custom delimiter. | Delimited files |
Quoting | MSQ | Used to specify modern style quoting which allows multi-line content. End-of-line characters must be enclosed with double quotation marks. Double quotation marks (“) appearing as the first or last character in field content will be interpreted as the start or end of multi-line content. | Delimited files |
Quoting | Standard | Used to specify standard quoting. Double and single quotation marks accepted only when they are the first and last blank character of a field value. | Delimited files |
Quoting | No Quote | Used if quotation marks are not to be accepted in the file. | Delimited files |
Character set | - | Select the character set for the data you are loading. Select a standard character set or a custom character set. | Delimited files |
Ignore end-of-file-character | - | Select to ignore characters with code point 26, which otherwise denotes an end-of-file character, are disregarded and can be part of a field value. Only relevant for .txt files | Delimited files |
Comment | - | Data files can contain comments between records. Comments are denoted by starting a line with one or more special characters, for example //. Specify one or more characters to denote a comment line. Qlik Sense does not load lines starting with the characters specified here. | Delimited files |
Loading data into your Qlik Sense app
Once you have finished selecting data, you can load your data into your Qlik Sense app. How you proceed will depend on whether you load data with Data manager or the Data load editor.
Data manager
To load data using Data manager, click Add data. This will open the data in the Associations view of the Data manager. In the associations view, you can continue to add data sources, transform the data, and associate the tables.
Data profiling is enabled by default when you click Data manager. Data profiling does the following:
- Recommends data associations.
- Auto-qualifies common fields between tables. This adds a unique prefix based on table name.
- Maps date and time fields to autoCalendar.
Tables are not associated on common field names automatically. You can associate tables in the Associations view.
If you want to load the data directly into your app, click and then disable data profiling.
When you add data with data profiling disabled, all existing data from data sources will be reloaded when you add the data. Tables will be associated on common field names automatically. Date and time fields will not be created.
Data load editor
To load data with the Data load editor, click Insert script once you are finished selecting the data. A load script is inserted into the script editor of the Data load editor. You can continue to edit the script in the script editor or you can click Load data to run the data load script.
Syntax: Loading a hosted file
LOAD *
FROM [lib://Google_Cloud_Storage1/myFile1.csv]
(txt, utf8, no labels, delimiter is ',', msq);
In this example, Google_Cloud_Storage1 is the name of the Google Cloud Storage connection.
Storing data from your Qlik Sense app in Google Cloud Storage
You can store table data into your Google Cloud Storage bucket in the Data load editor. You can either create a new load script or edit the existing script.
Do the following:
-
Using the SELECT command, specify a list of fields to be included in the upload.
-
Then write a Store command to set the file path in Google Cloud Storage.
-
Click Load data to run the data load script.
See the Store syntax for more detailed scripting instructions and supported file formats.
Limitations
- Folders must be specified in syntax with a trailing slash character: [ / ].
For example: LOAD * FROM [lib://Google_Cloud_Storage1/Path/Folder/] - The slash character [ / ] is the only character supported as a delimiter.
- Folders and files with the exact same name are not supported. It is not recommended to create files with a trailing slash.
-
Key names containing \ (backslash) are not supported.
- Wildcard/masking syntax is not supported. You may only load specified individual files, or all files in a specified folder.
- When using Qlik Sense Enterprise on Windows, the Include command is not supported.
Reference - Google Cloud documentation
You can refer to the Google Cloud API documentation to learn more about the requirements and restrictions imposed by the Google Cloud API.