Upgrading Qlik Data Gateway - Direct Access

New features and enhancements

Direct Access gateway 1.6.5 introduces the following features and enhancements:

• Support for the SAP ODP connector

  This version introduces the ability to access a SAP ODP source via Direct Access gateway. To facilitate this, a new Data gateway field has been added to the SAP ODP connector. The new field allows customers to choose which gateway to use to access their SAP ODP source.

  See also: SAP ODP Connector

• Extended logging capabilities

  This version introduces a new logger that Qlik Support can use to troubleshoot issues with specific gateway components.

• Newly supported region

  Japan is now supported as a Qlik Cloud tenant region.

• Performance enhancement

  Improved memory allocations for encrypted sessions.

Resolved issues

Direct Access gateway 1.6.5 resolves the following issues:

• Security vulnerability

  Secrets would be written to the DCAAS log when working with the ODBC connector.

• Process isolation performance

  Processes would take a long time to start under heavy load.

• Incorrect installation path

  Installing Direct Access gateway would always set Qlik as the root folder, even if the user chose a different folder in the setup wizard.

New features and enhancements

• Support for viewing and downloading log files from the Management Console

  This version introduces the ability to view Direct Access gateway log files in the Management Console and share them with Qlik Support if needed. Under normal circumstances, there is no reason to view the log files, but should you experience repeated failures or other issues, the log files will often help pinpoint the cause.

  See also: Viewing and downloading log files

• Support for installing Direct Access gateway silently

  Customers can now install Direct Access gateway silently, which is useful if you need to install Direct Access gateway on several machines throughout your organization.

  See also: Silently installing Direct Access gateway

• Improved session continuity

  When a reload session expired, a DirectAccess-1012 error would sometimes occur. Now, Direct Access gateway will try to keep the reload session alive by pinging Qlik Cloud more frequently.

Resolved issues

Direct Access gateway 1.6.4 resolves the following issues:

• Incorrect parameter name in the configuration.properties file

  When performing a new installation of Direct Access gateway, the EXTEND_FIRST_REQUEST_TIMEOUT parameter name in the configuration.properties file would incorrectly be called FIRST_REQUEST_TIME_EXTENSION.

• Reload pod availability

  When multiple reloads shared the same reload ID, the number of reload pods in QCS would reach the maximum.

Direct Access gateway 1.6.3 includes the following enhancements and bug fixes:

• Data loss prevention

  In previous versions, to prevent potential data loss, Direct Access gateway did not automatically reconnect to the data source during a reload. From this version, Direct Access gateway will now automatically reconnect to the data source as needed. To prevent data loss, the completed reload summary will be verified before reconnecting. If data loss is detected, the reload will fail.

• Timeout for reload sessions

  In rare scenarios, the reload session would stay alive indefinitely. To prevent such occurrences, this version introduces a three-hour timeout for reload sessions.

• Timeout extension for load requests

  Reloads would sometimes fail with the following errors:

  Connector error: The session does not exist (DirectAccess-1509)

  Connector error: The stream session expired for the load data request (DirectAccess-1012)

  To mitigate this issue, an EXTEND_FIRST_REQUEST_TIMEOUT parameter (for extending the load request timeout) was added to the configuration.properties file. The default is "0". Customers who frequently encounter these errors should try setting the timeout to 5-10 minutes.

• Support for detailed connection error messages

  Added support for viewing detailed connection error messages when using ODBC-based via Direct Access gateway connectors. Customers can now set the ShowErrorDetailMessage parameter in the Advanced connector settings to "true" to see the full error returned by the ODBC driver. The detailed error message will be show in the Data load editor.

  Information noteThe error message is provided in its entirety by the data source you are using. Qlik does not append any additional information to the error message or edit it in any way.

• Handling multiple reloads with the same reload ID

  From this version, the engine session ID will be used to distinguish between multiple reloads with the same reload ID. This will prevent potential conflicts, resulting in reloads failing with the following error:

  Connector error: data loading error (DirectAccess-4008

Direct Access gateway 1.6.2 resolves an issue that would sometimes cause the Connector Agent to crash when working in process isolation mode.

New features and improvements

Direct Access gateway 1.6.1 introduces several important features and enhancements, improving overall stability and performance.

• Support for spreading reloads across multiple connector processes

  Connector process isolation has been added to Direct Access gateway. This is an opt-in feature, meaning that it is off by default. Process isolation allows administrators with direct access to the gateway server operating system to control how many processes are available for reloads per connector type. Spreading reloads across multiple connector processes increases the level of isolation by reducing the number of reloads running concurrently on each connector process. This in turn can improve overall Direct Access gateway stability in cases where connector crashes have occurred.

  See also: Mitigating connector crashes during reload

• Added the ability to disable chunking when working with the SAP BW (via Direct Access gateway) connector

  This version introduces the BEX_ONE_CHUNK application parameter for the SAP BW (via Direct Access gateway) connector. When accessing SAP BW directly from Qlik Cloud (that is, not via Direct access gateway), to optimize performance, the result set is divided into chunks in the SAP backend and read as separate threads. However, when accessing SAP BW via Direct Access gateway, dividing the result set into chunks has no real advantage. On the contrary, it consumes more of the shared memory in SAP as it requires an additional temp table to be created for the duration of the data transfer. It's possible (though usually not required) to start chunking by setting BEX_ONE_CHUNK:false in the appsettings.json file, which is located in the SAP connector binaries folder.

• Diagnostics improvements for SAP BW (via Direct Access gateway) and SAP SQL (via Direct Access gateway) connectors

  For SAP BW (via Direct Access gateway) and SAP SQL (via Direct Access gateway) connector logs, a _port suffix was added before the log file name extension, and a ProcessID=### message was added to the connector's log files in order to distinguish between events from new instances of the connector listening on the same port.

• Reconnection improvements

  The reconnection mechanism during reload has been improved so that Direct Access gateway no longer needs to connect to the pod that started the reload (as was previously the case). Now, instead of repeatedly trying to connect to the pod that started the reload (which often resulted in reload failures), it will connect to the first available pod.

Resolved issues

Direct Access gateway 1.6.0 resolves the following issues:

• ODBC via Direct Access gateway connector log:

  When reloads were performed from different data sources, processes from one data source would erroneously appear in the logging of the other data source.

• Missing file in setup

  The configuration.properties file would not be copied during the Direct Access gateway installation.

• Reload error during Qlik Cloud resource scaling

  Qlik Cloud resource scaling would sometimes result in a DirectAccess-1006 error. An enhancement was made to improve handling of certain Qlik Cloud resource scaling scenarios that occur during application reloads performed via Direct Access gateway.

• Snowflake connection failure

  Connecting to Snowflake via the ODBC (via Direct Access gateway) connector would fail.

• Debugging application script failure in DLE

  Debugging application scripts in the Data Load Editor (DLE) with the Limit Results option enabled would sometimes fail with the following error:

  Connector error: Invalid load request (DirectAccess-1006).

Resolved issues:

• ODBC (via Direct Access gateway) connector: The connector would sometimes crash when canceling reloads of large tables. An improvement was made to help prevent this.

• Microsoft SQL Server (via Direct Access gateway) connector: When attempting to establish a connection to a SQL Server hosted on older Windows Server versions, the connection would fail due to a TLS compatibility issue and the following error would be returned:

  Please check the values for Username‚ Password‚ Host and other properties. Description: Client unable to establish connection - ERROR [08001] [Qlik][SqlServer] Problem establishing connection to the server.

  The SQL Server driver has been updated to correct the problem.

• SAP SQL (via Direct Access gateway) connector: When attempting to preview a table, the following error would sometimes occur:

  Exception of Type System.OutOfMemoryException

• SAP SQL (via Direct Access gateway) connector: Attempting to preview a table would sometimes time out after a few minutes with the following error:

  Request could not be completed.

• SAP SQL (via Direct Access gateway) and SAP BW (via Direct Access gateway) connectors: Added diagnostic information to the logs to assist in troubleshooting multiple concurrent requests.

New features and enhancements:

• Provides support for the new ODBC (via Direct Access gateway) connector.

• The Version column in Management Console will now display the Direct Access gateway version instead of the RepAgent (one of its sub-components) version.

Resolved issues:

• Upgrading Direct Access gateway would reset advanced ODBC Connector logging settings (such as log file retention and maximum size) to default.
• In rare cases, the ODBC connector would crash and could not be restarted by the Direct Access gateway agent.
• When a connection failed or the SQL query was not valid, a generic "Offset and length were out of bounds..." error would be returned instead of the actual error cause.
• In rare scenarios, hub reloads would fail with a "DirectAccess-4008" error for apps larger than 3 GB that had extended delays or processing after the initial gateway query. Additional logging has been added to assist in identifying any related issues.

Resolved issues:

• Changing the ODBC_LOG_LEVEL value in the C:\ProgramData\Qlik\Gateway\configuration.properties file would have no effect.

• Reload would fail with the following error if an active Direct Access gateway connection was not used for several minutes:

  Connector error: Invalid load request (DirectAccess-1006)

  The issue was resolved by not timing out if the session is still alive.

• Loading tables with numerous columns from MySQL and PostgreSQL would sometimes fail with the following error:

  System.InvalidOperationException: Failed to HPACK encode the headers

• When users did not have privileges on a Managed Space containing the Direct Access gateway, the following unclear error would be shown:

  Requested endpoint could not be provisioned due to failure to acquire a load slot: Object reference not set to an instance of an object.

  The error message was improved with a more accurate description of the issue.

• Multiple reload attempts would sometimes fail due to excessive memory consumption.

Features and enhancements:

Direct Access gateway checks its health periodically. Starting from this version, the health check will restart the Direct Access gateway service if it is unresponsive for five minutes.

Resolved issues:

• When starting a reload with a query that returns metadata immediately but only returns the first data chunk several minutes later, the following error would sometimes occur:

  Connector error: data loading error (DirectAccess-4008)

• In certain scenarios, Direct Access gateway would not reconnect to Qlik Cloud when a reload was interrupted.

• In rare situations, the following error would be encountered during reloads:

  No connection could be made because the target machine actively refused it.

• When connecting to SQL Server, special characters in passwords would not be encoded correctly, resulting in the following error:

  Command test connection returned non-success: Error message: Please check the values for Username‚ Password‚ Host and other properties. Description: Access credentials are not valid for this connection.

• The ODBC connector would sometimes crash when aborting load activity from the user interface while the query was returning data.

• In rare scenarios related to heavy loads, reloads would sometimes fail with the following error:

  Connector error: Unknown reason:. Index was outside the bounds of the array. (DirectAccess-1500)

Updated the CA certificates needed for authenticating the Qlik Cloud tenant.

This version resolves the following issue:

Qlik Data Gateway - Direct Access 1.3.1 is prone to excessive CPU utilization in certain scenarios. Version 1.3.2 resolves this issue and contains no other changes. To prevent CPU-related failures, we encourage all customers to upgrade to version 1.3.2 immediately.

This version includes the CA bundle for Qlik Cloud Government and resolves several issues.

CA bundle file for Qlik Cloud Government

The CA bundle authenticates the identity of the Qlik Cloud tenant, thereby ensuring a trusted connection.

In previous Direct Access gateway versions, the CA bundle file for Qlik Cloud Government customers was provided by Qlik as needed. From this version, the CA bundle file is included in the installation.

In a default setup, the CA bundle file is installed to the following location: C:\Program Files\Qlik\ConnectorAgent\caBundle]qcg_ca_bundle.pem

Resolved issues

• A security vulnerability was identified whereby the HTTP client used for proxy connectivity would follow a redirect.

• The Direct Access gateway agent would sometimes crash with the following error:

  Unable to write data to the transport connection: An established connection was aborted by the software in your host machine

• An exception would sometimes occur during log initialization, which would cause the Direct Access gateway to hang on restart.
• Reload session handling was improved to prevent unnecessary timeouts.
• The Direct Access gateway Version column in Management Console would indicate the wrong version.
• The agent would continue to send data to the gateway after the user stopped the task or the quota limit was reached.
• End-to-end encryption for Data Gateway would not be enabled by default and was controlled by runtime flags.

This version resolves the following issues:

• Version 1.2 of Qlik Data Gateway – Direct Access might be prevented from installing due to an issue with its digital signature.
• Configuration files would be overwritten during upgrade, resulting in changes being lost

Features and enhancements:

• Added support for SAP BW and SAP SQL sources. For information on setting up connectivity to these sources, see SAP NetWeaver.

Resolved issues:

• Improved stability and recoverability of the Qlik Data Gateway - Direct Access service, eliminating the need to manually restart the service
• Support for overwriting the default Direct Access gateway installation and configuration parameters

Features and enhancements:

• Added Windows Server 2022 support

Resolved issues:

• Resolved a potential stability issue related to choosing a custom path during setup

1.1.0

October 18, 2022

Features and enhancements:

• Support for Oracle Wallets
• Support for Google BigQuery
• Improved error messages

Resolved issues:

• Improved system recoverability after driver crashes
• The Qlik Data Gateway - Direct Access service would sometimes stop unexpectedly