Google Cloud Storage

The Qlik Google Cloud Storage Web Storage Provider Connector lets you fetch your stored data from Google Cloud Storage buckets, allowing you to stream data directly into your Qlik Sense app from your Google Cloud Storage account, just as you would from a local file. You can also export and upload compiled table data into your remote Google Cloud Storage.

When you authenticate the Google Cloud Storage Web Storage Provider Connector, you use an Identity and Access Management (IAM) user to provide access. This IAM user has the access configured in your Google Cloud Storage account. Refer to the Amazon documentation (https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-iam-policies.html) for guidance on determining an appropriate policy, including the ability to scope to specific files/permissions such as control over read/write. Any user granted Can Edit / Can Consume Data (Shared Spaces) or Can Consume Data / Can Manage (Managed Spaces) roles to the Google Cloud Storage data connection's space can view files in the configured S3 bucket as defined by the IAM user's policy.

Connecting to an Google Cloud Storage account

You need to select the connector in Qlik Sense, and authenticate the connector with your Google Cloud Storage credentials.

When selecting a Google Cloud Storage bucket, note that geographic distance between your Qlik Sense environment and the hosting may have an impact on download and upload performance.
For example: a Qlik Sense deployment in the North American region may have significantly slower download and upload speeds when connecting to a Google Cloud Storage bucket in the Asia-Pacific region.

You can access the connector through Data manager and the Data load editor.

Data manager: Click + and then Add new connection. Select Google Cloud Storage under Connect to new data source.
Data load editor: Click Create new connection and select Google Cloud Storage from the list.

Creating a connection

Authentication

You need to authenticate the connector with your Google Cloud Storage credentials:

Service account key file: You need one or more service accounts with appropriate roles and permissions in the IAM settings in the Google Cloud console. The simplest option is a service account with a storage admin role and no conditions. You will need to create a JSON key for each service account you wish to use. This key file should be downloaded and stored securely. Note that P12 keys are not supported, as they are considered legacy. In order to authenticate, you will need the key file and the bucket name you wish to connect to. These details are stored securely in the connection.
Bucket name: You can find bucket names in the Google Cloud console storage page.
Name: In Data load editor, you can specify a connection name or leave the default name for the connection. You cannot rename connections in Data manager.

In the Data load editor, you can verify the connection before you create it by clicking Test Connection.

When you are finished setting up your data connection, click Create. The connection credentials are verified automatically.

Connector location

The location of the connection depends on how you select data.

Data manager: It will be saved under File Locations.
Data load editor: It will be saved under Data connections.

Selecting data

When you create a Google Cloud Storage connection with Data manager, the data selection step follows immediately. When you create a data connection with the Data load editor, you need to open the data connection before you can select data.

Do the following:

Navigate to the file path location of your Google Cloud Storage account.
Locate your files by clicking on the folder names.

You can navigate backwards by clicking on the previous folder in the path field or by clicking . You can filter the file types that are shown by using the File type drop-down menu.
Select a file. Then, select the data tables and the data fields that you want to load.

You can rename data fields before loading them into Qlik Sense by clicking on the field name and entering a new name. The new name is mapped by Qlik Sense as an alias to that field name in the database.

Data file properties

If you are loading data from an Excel file or from a delimited file, you may need to adjust the data file properties. The following table describes the data file properties for these file types:

Data file properties available to be configured
Parameter	Option	Description	For file type
Field names	Embedded field names	Set to specify if the table contains Embedded field names or No field names. Typically, in an Excel spreadsheet, the first row contains the embedded field names.	Delimited files .xls/.xlsx
Field names	No field names	Select if the fields are not labeled. If you select this option, fields will be named A,B,C...	Delimited files .xls/.xlsx
Header size	-	Set to the number of header rows to ignore. Typically, you want to exclude rows that contain general information that is not in a columnar format.	Delimited files .xls/.xlsx
Delimiter	-	Set the field delimiter. Select a standard delimiter (comma, semicolon, space, tab) or a custom delimiter.	Delimited files
Quoting	MSQ	Used to specify modern style quoting which allows multi-line content. End-of-line characters must be enclosed with double quotation marks. Double quotation marks (“) appearing as the first or last character in field content will be interpreted as the start or end of multi-line content.	Delimited files
Quoting	Standard	Used to specify standard quoting. Double and single quotation marks accepted only when they are the first and last blank character of a field value.	Delimited files
Quoting	No Quote	Used if quotation marks are not to be accepted in the file.	Delimited files
Character set	-	Select the character set for the data you are loading. Select a standard character set or a custom character set.	Delimited files
Ignore end-of-file-character	-	Select to ignore characters with code point 26, which otherwise denotes an end-of-file character, are disregarded and can be part of a field value. Only relevant for .txt files	Delimited files
Comment	-	Data files can contain comments between records. Comments are denoted by starting a line with one or more special characters, for example //. Specify one or more characters to denote a comment line. Qlik Sense does not load lines starting with the characters specified here.	Delimited files

Loading data into your Qlik Sense app

Once you have finished selecting data, you can load your data into your Qlik Sense app. How you proceed will depend on whether you load data with Data manager or the Data load editor.

Data manager

To load data using Data manager, click Add data. This will open the data in the Associations view of the Data manager. In the associations view, you can continue to add data sources, transform the data, and associate the tables.

Data profiling is enabled by default when you click Data manager. Data profiling does the following:

Recommends data associations.
Auto-qualifies common fields between tables. This adds a unique prefix based on table name.
Maps date and time fields to autoCalendar.

Tables are not associated on common field names automatically. You can associate tables in the Associations view.

Managing data associations

If you want to load the data directly into your app, click More and then disable data profiling.

When you add data with data profiling disabled, all existing data from data sources will be reloaded when you add the data. Tables will be associated on common field names automatically. Date and time fields will not be created.

Data load editor

To load data with the Data load editor, click Insert script once you are finished selecting the data. A load script is inserted into the script editor of the Data load editor. You can continue to edit the script in the script editor or you can click Load data to run the data load script.

Syntax: Loading a hosted file

LOAD *
FROM [lib://Google_Cloud_Storage1/myFile1.csv]
(txt, utf8, no labels, delimiter is ',', msq);

In this example, Google_Cloud_Storage1 is the name of the Google Cloud Storage connection.

Storing data from your Qlik Sense app in Google Cloud Storage

You can store table data into your Google Cloud Storage bucket in the Data load editor. You can either create a new load script or edit the existing script.

Do the following:

Using the SELECT command, specify a list of fields to be included in the upload.
Then write a Store command to set the file path in Google Cloud Storage.
Click Load data to run the data load script.

See the Store syntax for more detailed scripting instructions and supported file formats.

Limitations

Folders must be specified in syntax with a trailing slash character: [ / ].
For example: LOAD * FROM [lib://Google_Cloud_Storage1/Path/Folder/]
The slash character [ / ] is the only character supported as a delimiter.
Folders and files with the exact same name are not supported. It is not recommended to create files with a trailing slash.
Key names containing \ (backslash) are not supported.
Wildcard/masking syntax is not supported. You may only load specified individual files, or all files in a specified folder.

Reference - Google Cloud documentation

You can refer to the Google Cloud API documentation to learn more about the requirements and restrictions imposed by the Google Cloud API.

Google Cloud API Introduction

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here