Making data request with synchronous, asynchronous, batch, and auto mode

You can make the following request types with Qlik Web Connectors:

• Synchronous
• Asynchronous
• Batch
• Auto

Writing requests

For/next loops are often used to run a large number of API requests with Qlik Web Connectors. These modes let you extract the maximum performance from Qlik Web Connectors and minimize load time of the QlikView or Qlik Sense application.

For example, if you run a large number of text messages through one of the available sentiment analysis connectors, like Sentiment140, it can result in a long load time.

Example:  

In the script example above, each message is processed sequentially. This means that the sentiment score for the next message cannot be calculated until the current request has completed. If each request takes 250 ms, then it will take 40 minutes to score the sentiment score of 10,000 Tweets.

Many APIs allow multiple requests to be made in parallel from the same client so we can use different request types to improve the performance of the Qlik Web Connectors.

Synchronous requests

Synchronous requests are the default mechanism and all tables of all connectors support this. A request to Qlik Web Connectors results in one or more sequential requests to the underlying API and this is the script you receive when copying the standard generated script from Qlik Web Connectors.

For simple for/next loops, It is possible to simplify the script by using a temporary table.

Example: For/next loop

Example: Temporary Params table

Create a temporary table named Params that contains the same parameters that are passed in each iteration of the for/next loop above.

There are several important changes to the new script:

• An additional column named rowKey, which is set to the Tweet ID, so we can link it back into the data model when the results come back.
• The Params table is saved as a CSV file. It is important that the user account that is running the Qlik Web Connectors has read access to this file.

The script makes a request to Qlik Web Connectors but instead of passing the usual parameters, it passes the path of the previously created CSV file using the processParamsSync parameter. We also read back the rowKey and alias ID back to the Tweet ID.

You can apply this method to almost any Qlik Web Connectors request, and it is the basis of the Asynchronous, Batch and Auto modes.

Asynchronous requests

To make asynchronous requests, you simply change the parameter name processParamsSync into processParamsAsync:

Qlik Web Connectors attempts to run the requests on a number of parallel threads and the above call only completes when all the requests have finished.

Specifying the number of threads

By default, Qlik Web Connectors use a maximum of 30 parallel threads.

You can set an alternative value, between 1 and 50, by including a maxThreads=VALUE parameter in the request.

Batch requests

To make batch requests, you simply change the parameter name processParamsSync into processParamsBatch:

Qlik Web Connectors attempt to run the requests using the Batch feature that the API provides, and the above call only completes when all the requests have finished.

Auto requests

If you want Qlik Web Connectors to decide the best method, you can change the parameter name processParamsSync into processParamsAuto:

When you use auto requests, Qlik Web Connectors uses the following rules:

• If the table supports batch mode, Qlik Web Connectors uses this method.
• If the table explicitly supports asynchronous mode, Qlik Web Connectors uses this method.
• Otherwise, Qlik Web Connectors uses the standard synchronous mode.

Using URLs instead of file paths

When using a processParamsxxx parameter, as well as passing a file location to it, you can also pass a URL beginning with http or https.

Special parameters

Certain parameters can be used by any Qlik Web Connectors connector. These parameters should not be placed in the Params CSV file but specified in the request URL for the table.