Working with Qlik Cloud Analytics
Qlik Cloud Analytics is far more than just an analytics tool, it's an end-to-end environment for your analytics. By familiarizing yourself with the Qlik Cloud Analytics environment, you will be able to structure your analytics workflow to meet your data, governance and manageability needs.
Tenants, user roles and entitlements
Tenants
Each customer creates an instance of the Qlik Cloud environment, which is called a tenant. Qlik Cloud Analytics is a service of Qlik Cloud and therefore each Qlik Cloud Analytics customer has their own tenant. Tenant names are automatically generated for the region that you choose on creation, for example abcdefgh.eu.qlikcloud.com. Customers can optionally set their own alias, such as mycompany.eu.qlikcloud.com.
Roles
There are several roles available in Qlik Cloud Analytics, which determine the functions that a user can perform. These compliment the platform roles such as Tenant Admin. The roles available in Qlik Cloud Analytics are:
Analytics Admin – A user with this role is a partial administrator. They have access to the the Management Console, but only to the areas of governance and content.
Shared Space Creator – A user with this role can create shared spaces.
Managed Space Creator – A user with this role can create managed spaces.
Private Analytics Content Creator – A user with this role can create analytics content in their personal space, and create alerts and subscription reports.
Steward – A user with this role can create and maintain business glossaries.
These Qlik Cloud Analytics roles work with the platform roles such as Tenant Admin and the users’ entitlements, such as professional and analyzer access. Roles can be assigned to groups or to individual users.
Spaces
Spaces are areas of the Qlik Cloud Analytics hub used to develop and control access to apps, data files and data connections. The types of spaces in Qlik Cloud Analytics are:
Personal spaces – Private work areas for each user in the Qlik Cloud Analytics hub.
Shared spaces – Areas used to develop apps collaboratively and share them with other users in the space. A group of users may use a shared space for the private development and consumption of their own apps.
Managed spaces – Governed sections of the Qlik Cloud Analytics Hub that are used for providing access to apps with strict access control both for the app and the app data.
In addition to the analytics spaces, Qlik Talend Data Integration provides:
- Data spaces - Used to manage access in Qlik Talend Data Integration. Files loaded into a data space by Qlik Talend Data Integration can be consumed by Analytics apps.
Data files and data connections, just like apps, can also be stored and reused across the different space types. These assets can be accessed from other spaces if the appropriate permissions have been granted. For example, an application in the Sales managed space may use a data connection defined in the Sales shared space if desired.
Access to shared and managed spaces can be assigned directly to either users or to groups. It is considered best practice to use groups and manage group membership in the identity provider rather than directly assign access to individual users. Various levels of access can be assigned, so it is possible that a user can have different access permissions to different spaces.
Enterprise data at scale
Tenant resources
Each Qlik Cloud Analytics tenant provides a base capacity for analytics apps with the ability to increase this if required. The subscription type determines the base capacity available. With a user-based subscription, each Qlik Sense app can consume up to 5 gigabytes of memory, while a capacity-based subscription provides up to 10GB. Qlik Cloud Analytics scales to meet the demand that is required by user demand with no configuration needed.
Case study — Performance impact as data volumes and users increase
At Qlik, we have a dedicated team focused on the performance and scalability of our platform. As part of that process the team benchmarks typical scenarios that our customers face to ensure Qlik Cloud Analytics performs at scale.
Recently, we benchmarked a typical sales app, looking at the performance with transaction volumes of 10, 50 and 100 million records. We evaluated how the application performed as the number of users increased over time and as we can see, performance actually improves with more users due to intelligent caching by the engines.
To achieve these results, Qlik Cloud automatically monitors the resource usage in the engines and allocates more engines as needed, ensuring adequate resources are always present to provide for user demand. When this occurs there may be a temporary increase in response times as the cache in the new engine needs to populate, however the impact of this is relatively short and the performance soon stabilizes.
Behind the scenes, we can see Qlik Cloud allocating those engines as needed. There are no limits on the number of engines that can be allocated at the app or customer level and no manual intervention from either the customer or Qlik is required to achieve this. These tests were run in a standard production Qlik Cloud tenant in the US. While these tests are reflective of how Qlik Cloud operates for all users and applications, actual results will vary based on the individual application and usage patterns.
Large app support
Certain use cases may require apps that use more memory than is provided as standard. This is referred to as large app support. This capability is sold in packs of 20GB and 100GB. The capacity allocated to your tenant determines the maximum app size you are able to utilize in a single app up to a maximum size of 50GB. Reloads sometimes use more memory temporarily, and this is taken into account as part of large app support. Should high user volume require more resources, additional engines will be allocated to handle this load.
If a customer is allocated a single 20GB pack, they can open an app up to 20GB, and with 2 packs either an app of 40GB, or two apps of 20GB. Purchasing over 50GB, either in 20GB or 100GB packs, does not increase the maximum size of a single app beyond 50GB, but does allow multiple large apps to be opened simultaneously. If an app is not opened and is not being reloaded it will not consume this, so customers only need sufficient resources for their concurrent usage.
Expanded storage options
Qlik Cloud Analytics supports integration with enterprise storage solutions such as SFTP, AWS S3®, Google Cloud Storage® and Azure Storage® to enable access to a customer's own storage assets. We also support a number of consumer file storage solutions such as DropBox® and Google Drive®.
A common scenario for external storage is sharing content between multiple Qlik Cloud tenants, or between a client managed environment and your Qlik Cloud tenant. For example a customer who is using Qlik Sense Client managed to generate QVDs, could write those to external storage which is then consumed by you Qlik Cloud tenant. Over time the client managed applications generating QVDs could be moved to SaaS without having to change the downstream applications reading the QVDs.
Automatic scaling to meet user load
When using client-managed deployments on-premises or in a public cloud , customers are required to estimate and size infrastructure for their peak usage. This is a complex process and often customers either under or over resource for peak usage times, leading to poor performance or unnecessary cost. Often these resources are only needed for very short periods to deal with daily or weekly peaks.
Qlik Sense Enterprise SaaS removes the complexity and costs of estimating, procuring, running and managing infrastructure because it is included in the subscription. More importantly, user satisfaction is higher as dealing with increased load is instantaneous, not subject to delays of procurement, installation, and configuration of infrastructure.
When a user accesses an application in Qlik Sense Enterprise SaaS, we initially check if the application is already open on one or more engines. If it is not, or these engines are already under heavy load, Qlik responds in turn by providing additional compute engines dynamically and opening another copy of the application. This autoscaling requires no configuration, management or extra expense from the customer and is transparent to the user. When the resources are not needed, Qlik with reduce the number of copies of the application open. This is true for both standard and large apps.
Managing data assets
Qlik has introduced cataloging capabilities through the catalog tab in the top menu of the Qlik Sense Hub. Switching to the catalog browser view allows users to create and easily navigate spaces, manage space permissions, and view and find content from one place. Enter keywords and/or select filters to locate charts, apps, data, notes, and links. Users can also view the content by owner creator and tag making it easier and faster to manage content and find what is needed, when it’s needed.
Searching and filtering can be done from the top menu or for additional detail, a vertical pop-out panel is available. A primary benefit of the design is the ability to multi-select the type of objects that users want to filter. Look soon for catalogued master items, business models, and more. Lastly, the Catalog content browser has been developed as a micro-component. Look for this rich search and browsing experience within Data manager of a Qlik Sense application.
Data lineage and impact analysis
The data lineage process identifies and documents data from its source to target, noting any changes as it moves through your data lifecycle. Using Qlik’s lineage connectors, Qlik automatically finds and documents any relationships between datasets and across multiple BI tools. This lineage provides visibility on the source and journey of each dataset.
Qlik lineage connectors expand Qlik impact analysis capabilities to include traceability information from multiple BI tools and data sources.
Qlik supports lineage information from Qlik’s client managed products, some third party BI tools, data files, and databases.
Field-level lineage allows for detailed investigation into how fields have been calculated and their specific origin across transforms and applications. While you explore the lineage, you can interactively change the base node to another table, application, or field on the screen to focus your investigation.
Downstream lineage is called impact analysis because it analyzes which objects will be impacted by changes to your data; these objects are the dependents of the base node. Qlik Cloud provides information and counts by type of dependent objects in a summary view.
Business users examining a given field will have an aggregated summary of downstream impact that delivers insight into:
Which object types would be impacted by a change to this field including databases, file storage, apps, and links
What is the number of direct dependencies and indirect dependencies by type
Who are the owners of the items that are impacted if you make a change
Move apps through a development lifecycle, not development servers
In a traditional BI environment apps would be developed on a development server. Once development was complete, they would be moved to a test server. Issues found in testing would mean several iterations of this process until the application could be deployed to production, requiring a lot of resources and infrastructure to manage.
With Qlik Sense Enterprise SaaS, apps are stored in discrete spaces. Each space has its own security settings, data connections and file storage. Customers can create as many development, test, and production spaces as needed to suit their software development lifecycle. This approach allows much greater flexibility, agility and reduced infrastructure expense than with a traditional on-premises setup.
In the above example, users consume the app in the production space. When a change to the app is requested, a copy is made in the development space and is published to the test space when ready for app testers to review it. Several cycles may occur until the app is ready to be released to production. To facilitate these flows, each space has its own data connections and file storage, so that an app will load the appropriate data for the applicable life-cycle phase.
Integrating data with Qlik Sense Enterprise SaaS
Qlik understands that while many organizations are moving their systems to the cloud, there will always be some systems such as mainframes, that remain on-premises. Additionally, some customers choose to migrate some systems to private clouds that are not directly accessible from public SaaS environments. Consequently, Qlik Sense Enterprise SaaS provides several solutions to integrate with a customer's data, whether that is on-premise, or in the private or public cloud.
Qlik's Connector Factory
Straightforward access to a customer's data is a key requirement for analytics. In a quickly changing market with many new players, especially in the cloud space, adding support for these sources in a timely manner is a key requirement. To address this Qlik has introduced Connector Factory. This initiative within Qlik leverages our own expertise as well of that of our technology partners to close these gaps as soon as possible for our customers. Through this initiative, Qlik plans to deliver over 100 new connectors in 2023.
Working with files
While we will often load data directly from a database or cloud system directly, there are times when we need to work with files. Qlik supports many common file formats such as Excel (XLSX), CSV, XML and Parquet. Files can be read directly from Qlik Cloud's own data file storage, or from our file connectors such as SFTP, S3, Azure Storage, and Google Cloud Storage, to name a few. Qlik also supports its own format, QVD, which is highly optimized for fast loading.
Qlik apps also allow files to be written to Qlik Storage and to file connectors. This can be useful where data is to be used for multiple applications, and where we wish to create curated data sets for use by others. Files can be stored in text, QVD, or Parquet formats.
Qlik Data Gateway - Direct Access
Businesses commonly enforce strict firewall policies that block external access to their data sources, whether on-premise or in a private cloud deployment. This presents a challenge when such businesses need to access their data from the cloud for analytics.
In the past customers had to choose between pushing this data to Qlik Cloud using tools such as Qlik Data transfer or Qlik Talend Data Integration (see below) or opening up connectivity to these systems to Qlik cloud through their firewalls. For many use-cases none of these options are desirable.
Qlik has introduced the Qlik Data Gateway for direct access to overcomes this challenge by eliminating the need to open inbound firewall ports.
Qlik Data Gateway for direct access allows Qlik Sense SaaS applications to securely access on-premises data via a data gateway. The gateway is installed in a customers data center or private cloud and initiates an Outgoing secure reverse tunnel from the gateway to Qlik Cloud via an HTTPS (SSL 1.3) connection. Qlik cloud is then able to directly connect to the customers data sources in real time without any requirement to expose these data sources externally. Authentication for the gateway is managed though a JSON Web token (JWT) generated by the client which must then be stored in Qlik Cloud. Authorization for the gateway is managed by assigning a gateway to a space, then controlling access though authorization on that space.
Gateways should be deployed physically close to the data sources being consumed. Customers can set up as many gateways as they require. This allows customers to support different parts of their organization, whether separated by geography or security policy. Gateways do not need to be able to contact each other (and are not aware of each other); they only need to be able to connect to the data sources and to make an outbound connection (though a web proxy if required) to Qlik Cloud.
As well as most of our standard SaaS Data connectors, Qlik Data Gateway - Direct Access supports ODBC connections on the host where the gateway is installed. This allows access to any data source a customer has an ODBC driver for, even sources Qlik does not provide explicit support for that data source.
Qlik DataTransfer
Qlik DataTransfer is a lightweight utility that is included in Qlik Sense Enterprise SaaS and securely pushes on-premises data and local files to the cloud. It is designed for customers who do not require either a full data integration solution such as Qlik Catalog, or a full Qlik Sense Enterprise Client-Managed deployment, and do not wish to open firewalls to connect to on premise data. Qlik DataTransfer provides:
Connection creation and selection dialog to pick source data from standard connectors such as ODBC and REST
Ability to create datasets which can run manually or on a schedule to connect to the source data and land this in Qlik Sense Enterprise SaaS, and optionally reload an app within the same SaaS space as part of the schedule
A “folder watcher” which will also move supported file types (e.g. QVD, CSV) to the data files area within the space of choice and optionally reload an app within the same SaaS space when triggered.
Capability to reload pre-built Qlik apps on-premises and move these to the SaaS tenant
Data is uploaded to Qlik Sense enterprise SaaS securely over HTTPS and stored in the customer’s tenant using encryption at rest.
Qlik Talend Data Integration
Qlik Talend Data Integration is Qlik’s hosted and managed Integration Platform as a Service (iPaaS). Our vision is to provide a broad variety of data integration services aimed at helping users move from passive to active BI. A brief summary of Qlik Talend Data Integration is provided below. For more information, see the Qlik Talend Data Integration evaluation guide.
Qlik Talend Data Integration continuously streams data in near real-time from on-premises systems, such as relational databases, mainframes, and SAP systems to the Qlik Cloud tenant. QVDs created by the services are automatically updated whenever the source data changes and are ready for consumption by analytics apps without the need for scripting.
Qlik Talend Data Integration consists of:
Landing - Landing transfers data from the customer's source to target environments. Landing can be either via the data movement gateway, or direct from supported cloud sources. Where the target is Qlik Cloud, this must be landed via Amazon S3.
Qlik Data Movement Gateway - The Qlik Data Movement gateway is a lightweight agent which runs in a customer's environment replicating data to a target environment. Targets could be Qlik Cloud (where data is published to QVD files for analytics consumption), or cloud storage, Data warehouses or Data Lakes.
Cloud sources use Qlik Connector technology to directly access cloud sources for ingestion into a customers targets.
Storage - The storage data task consumes the data that was landed to the cloud landing area to generate datasets ready for use or transformation.
Transformation - Transformation provides row level actions against your data set and allows you to create reusable transformation data tasks. Transformations can create materialized tables or views depending on your requirements. Simple transformations could be used to concatenate columns, run functions against a column or rename columns. For more complex operations SQL based transformations can be used.
Data marts - Data marts can be created from either storage or transform tasks, or a combination of both. Data marts are star schema based.
Qlik Data Integration Platform (client-managed)
Qlik’s Data Integration Platform is a bundled solution with components such as Qlik Catalog, Qlik Replicate,and Qlik Compose for Change Data Capture Streaming, Data Lake Creation, and Data Warehouse Automation, all providing the ability to publish data from a wide breadth of data source end points to cloud storage locations, such as S3 buckets and Cloud data warehouses. These can then be used by Qlik Sense Enterprise SaaS to load apps. Data Integration products which can be used with Qlik Sense Enterprise SaaS include:
Qlik Replicate - Universal data replication and real-time data ingestion
Qlik Catalog - A secure, enterprise-scale data catalog to easily find, prepare and deliver analytics-ready data
Qlik Compose - Agile data lake creation and data warehouse automation
For more information on Qlik’s Data Integration Platform, see: Qlik’s Data Integration products.
API and script-based data integration
It is possible to move applications to and from the Qlik Cloud platform using either our APIs directly, or via the qlik-cli tool. Applications can be created programmatically, imported, published, and reloaded. They can also be exported with or without data, and many other operations.
Qlik Sense Enterprise Client-Managed for Windows
Qlik Sense Enterprise on Windows provides the ability to automatically synchronize Qlik Sense apps on-premises with Qlik Sense Enterprise SaaS. It is possible to use rules to tag which apps are distributed to Qlik Sense Enterprise SaaS, so only a subset of apps needs to be distributed. This feature allows customers to deploy some apps to Qlik Sense Enterprise SaaS and keep other apps on-premises and/or in a private cloud.
QlikView publisher
QlikView provides the ability to publish QlikView apps to Qlik Sense Enterprise SaaS through QlikView publisher. This capability allows QlikView customers to significantly reduce infrastructure costs while continuing to reload their QlikView apps on-premises.
Did this page help you?
If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!