QlikWorld 2020 Global Conference. Join us to discover how to get the most from your data. Act fast. Register now and save.

Amazon S3 Metadata

The Qlik Amazon S3 Metadata connector uses the Amazon S3 API to access your Amazon S3 metadata, such as the names of files and subfolders in your Amazon S3 bucket.

Note: The Amazon S3 Metadata connector is only available on cloud editions of Qlik Sense.

Ways to access your data

To access your Amazon S3 metadata, you need to authenticate the connector with your Amazon S3 account credentials: access key, secret key, region, and bucket name.

Creating a connection and selecting data

Loading data from tables

After you authenticate the connector with your account credentials, use the tables to fetch your data. Below are use cases for some of the available tables:

Table Description
ListBuckets Fetches a list of buckets from your Amazon S3 account.
List Fetches a list of folders in a bucket and the objects inside those folders. You need the bucket region and bucket name to run this table.
ListObjects Fetches a list of objects in a bucket. You need the bucket region and bucket name to run this table.
Metadata Returns metadata for the object.
InProgressUploads Fetches a list of uploads currently in progress.

To select and load data from a table, enter the required table parameters and click Preview data. Required parameters are marked with an asterisk (*). The table fields are displayed under the Data preview tab. You can select fields individually by selecting the box beside each field name. Select Insert script after you have made your selection.

Creating a connection and selecting data

Reference - Amazon S3 documentation

You can refer to the Amazon S3 API documentation to learn more about the requirements and restrictions imposed by the Amazon S3 REST API.

Amazon S3 REST API Introduction

Troubleshooting

You receive generic access error messages

There are multiple generic access error messages: "cannot open file", "no fields to load", etc. that can occur.

Possible cause  

You have incorrectly entered Amazon S3 credentials or file/bucket information.

Proposed action  

Ensure that your Amazon S3 account credentials have been correctly entered:

  • Check that your Amazon S3 access key, secret key, region and bucket name are accurately entered in your connector configuration.
  • Check that your load script correctly references your connection name, file name, file path, and file formatting information.
  • Ensure that the file is available in your Amazon S3 instance at the specified file path.

You receive an error message that you have reached the API rate limit

Possible cause  

You have exceeded the API limits that are imposed on Qlik Web Connectors by the Amazon S3 API.

Proposed action  

To reduce the impact of reaching the API rate limits, develop your app with the following in mind:

  • Extract only the data you need.
  • Reload one Amazon S3-based application at a time.
  • Ensure that loops in your script that make API calls will not result in infinite loops.

Did this information help you?

Thanks for letting us know. Is there anything you'd like to tell us about this topic?

Can you tell us why it did not help you and how we can improve it?