Skip to main content Skip to complementary content

System requirements for Qlik Big Data Index deployment

Qlik Big Data Index is deployed in a Virtual Private Cloud (VPC) using Kubernetes. You need access to the following hardware, software and information to be able to prepare the environment for deployment.

Remote access PC

A PC with deployment administrator access to:

  • Accounts needed to manage the deployment environment.
  • The VPC where you want to deploy the QBDI cluster.

The PC must have the following software installed and configured with PATH to execute from a command window.

  • Kubectl (Kubernetes CLI)
  • jq
  • Helm 2.9.0 or Helm 3

Kubernetes cluster

You need a Kubernetes cluster setup with internet access to Qlik Helm and container image repository. The following environments are supported:

  • Amazon Web Services (AWS) 1.9.7 using Amazon Elastic Container Service for Kubernetes (Amazon EKS).

    You can use EKSCTL to set up the cluster.

  • Microsoft Azure 1.9.1 using Azure Kubernetes Service (AKS).

    You can use Azure CLI to set up the cluster.

  • Google Cloud Platform using Google Kubernetes Engine (GKE).

    You can use Google Cloud SDK to set up the cluster.

  • Non-managed Kubernetes 1.9.4 or 1.10.3 environment deployed with Bare Metal.
  • Amazon Web Services (AWS) 1.9.7 with Kubernetes deployed with KOPs.
  • For test, evaluation or development purposes, you can use:

    • minikube for Windows with Kubernetes 1.9.4.
    • Docker for Mac or Windows with Kubernetes 1.9.4 enabled.

Data source files

You can use non-nested Parquet 2.0 files or Optimized Row Columnar (ORC) files as data source for creating the index. For more information about supported data types, see Preparing data source files.

Storage

Depending on the cardinality of the original data, the total size of indexed data can be up to 1.6x the size of original data.

The following file systems are supported:

  • Amazon S3
  • HDFS 2.6.5
  • HDFS 2.7.5
  • HDFS 2.9.0
  • HDFS 3.0.0
  • Amazon EFS (for AWS deployment)
  • Azure Files (for Microsoft Azure deployment)
  • Google Cloud Storage (for Google Cloud Platform deployment)

Cluster size

For optimal performance and cost-effective cluster setup, the ratio between number of indexlets to number of cores in total across nodes in the cluster should be ranging from 2:1 to 5:1.

RAM

Qlik Big Data Index exploits caching of results from inference and hypercube operations for optimal performance. Each node should have minimum 120GB RAM per 1 billion rows.

Example cluster

An example of cluster size for TPCH_1000:

  • Number of rows: ~10 B → ~600 indexlets
  • Number of nodes: 6
  • vCPU per node: ~20 - 50 to handle ~100 indexlets
  • RAM per node: ~250 GB

CPU requirements

The CPU must support AVX2 and BMI2 instruction sets.

Clients

The following clients are supported for Qlik Big Data Index:

Qlik Sense Enterprise on Windows June 2020

Licensing requirements

Information noteLicense activations request access to the Qlik Licensing Service. Open port 443 and allow outbound calls to license.qlikcloud.com.

Use of a proxy is supported.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!