Skip to main content

Create an Apache Spark connection

To access your data stored on an Apache Spark database, you will need to know the server and database name that you want to connect to, and you must have access credentials. Once you have created a connection to an Apache Spark database, you can select data from the available tables and then load that data into your app or document.

In Qlik Sense, you connect to an Apache Spark database through the Add data dialog or the Data load editor.

In QlikView you connect to an Apache Spark database through the Edit Script dialog.

Setting up the database properties

Database properties that can be configured
Database property Description Required
Spark Server Type The type of Spark server can be Shark Server, Shark Server 2, or Spark Thrift Server. yes
Host The IP address or host name of the Apache Spark server. yes
Port Server port for the Apache Spark database. yes
Database The name of the Apache Spark database. yes

Authenticating the driver

Qlik Sense: Apache Spark authentication properties

Authentication properties that can be configured
Property Description
Mechanism Authentication with user name only, with user name and password, or with no authentication. If the Spark Server Type is Shark Server, you must select No Authentication. If the Spark Server Type is Spark Thrift Server, most configurations require User Name authentication. Selecting User Name and Password or User Name gives you the option to set up Account properties.
Username User name for the Apache Spark connection.
Password Password for the Apache Spark connection.
Name

Name of the Apache Spark connection.

The default name will be used if you do not enter a name.

QlikView: Apache Spark authentication properties

Authentication properties that can be configured
Property Description
Mechanism Authentication with user name only, with user name and password, or with no authentication. If the Spark Server Type is Shark Server, you must select No Authentication. If the Spark Server Type is Spark Thrift Server, most configurations require User Name authentication.
Username User name for the Apache Spark connection.
Password Password for the Apache Spark connection.

Account properties

Credentials

Credentials are used to prove that a user is allowed to access the data in a connection.

There are two types of credentials that can be used when making a connection in SaaS editions of Qlik Sense. If you leave the User defined credentials check box deselected, then only one set of credentials will be used for the connection. These credentials belong to the connection and will be used by anyone who can access it. For example, if the connection is in a shared space, every user in the space will be able to use these credentials. This one-to-one mapping is the default setting.

If you select User defined credentials, then every user who wants to access this connection will need to input their own credentials before selecting tables or loading data. These credentials belong to a user, not a connection. User defined credentials can be saved and used in multiple connections of the same connector type.

In the Data load editor, you can click the Primary key underneath the connection to edit your credentials. In the hub or Data manager, you can edit credentials by right-clicking on the connection and selecting Edit Credentials.

Account properties that can be configured
Account property Description
User defined credentials Select this check box if you want users that access this connection to have to input their own credentials. Deselect this check box if credentials can be shared with anyone who has access to this connection.
New credentials Drop-down menu item that appears if User defined credentials is selected.
Existing credentials Drop-down menu item that appears if User defined credentials is selected.
User User name for the connection.
Password Password for the connection.
Credentials name Name given to a set of user defined credentials.

Setting SSL options

SSL options that can be configured
Property Description Required
Enable SSL Require SSL authentication on the connection the Apache Spark server. No
Use System Trust Store Use the SSL certificate located in the standard system location used for storing trusted certificates. Yes, if certificate is stored in the standard system location. Available only when SSL is enabled.
Trusted Certificate The full path to the SSL certificate if it is not stored in the standard system location. Yes, if certificate is not stored in the standard system location. Available only when SSL is enabled.
Allow Self-signed Server Certificate Accept an SSL certificate from the server that is self-signed and not verified by a trusted authority. No. Available only when SSL is enabled.
Allow Common Name Host Name Mismatch Allow a mismatch between the SSL certificate's common name and the name provided in Host name field. No

Miscellaneous properties

Miscellaneous properties and options that can be configured
Property Description
Query timeout Amount of time before a data load query times out. Can be set from 30 seconds to 65535 seconds. Default is 30 seconds.

Load optimization settings

Load properties that can be configured
Property Description Required

Enable Bulk Reader

Select this to include larger portions of data in the iterations within a load. This may result in faster load times for larger datasets. If not selected, data will be loaded row by row. No
Max String Length

Maximum length of string fields. This can be set from 256 to 16384 characters. The default value is 4096. Setting this value close to the maximum length may improve load times, as it limits the need to allocate unnecessary resources. If a string is longer than the set value, it will be truncated, and the exceeding characters will not be loaded.

No

Using Advanced options

Advanced options that can be configured
Option Description Required

Name

Name of additional properties.

You can add more than one additional properties.

no
Value

Value of additional properties.

You can add more than one additional properties.

no
Thrift Can be set to Binary, SASL, or HTTP. Default = SASL yes
Note: When you connect to an Apache Spark database with the Data load editor or the Edit Script dialog, Test Connection enables you to test the connection before you attempt to create it.