Managing big data with on-demand apps

On-demand apps enable you to load and analyze big data sources in Qlik Sense Enterprise. Trying to analyze an entire big data store at one time is highly inefficient. Nevertheless, to make representative visualizations, all the data must be discoverable. Qlik Sense on-demand apps give users aggregate views of big data stores and allow them to identify and load relevant subsets of the data for detailed analysis.

On-demand apps expand the potential use cases for data discovery, enabling business users to conduct associative analysis on larger data sources. They allow users to first select data they are interested in discovering insights about and then interactively generate an on-demand app with which they can analyze the data with the full Qlik in-memory capabilities.

On-demand app components

Qlik Sense manages the loading of big data sources with selection apps that provide aggregated views of the big data and also enable a user to zoom in and analyze finer-grained data. Embedded in each selection app are on-demand app navigation links to one or more template apps used as the basis for creating on-demand apps. Properties of the navigation links and template apps enable you to tightly control the shape and volume of data loaded into on-demand apps.

Apps can be generated repeatedly from the template app to track frequently changing data sets. While the data is filtered according to selections made in the selection app, the on-demand app content is dynamically loaded from the underlying data source. The same on-demand app can be generated multiple times to make fresh analyses of the data as they change.

Constructing on-demand apps

Because on-demand selection and template apps require special load scripting, they are usually created by users with experience writing Qlik Sense load scripts. On-demand selection apps, for example, must load data with a modest level of dimension granularity. On-demand template apps contain load scripts with data binding expressions used to formulate the queries made on the data sources.

A selection app can be linked to multiple template apps, and a single template app can be linked to by multiple selection apps. But the template app's data binding expressions must correspond to fields in the selection apps that link to it. For that reason, selection and template apps tend to be created in conjunction with one another and often by the same experienced script writer.

Creating navigation links also requires an understanding of the fields in the selection app that have corresponding bindings in the template app. That is because each navigation link requires an expression that computes the total number of detail records. That total represents the aggregate records accessible by way of the selection state in the selection app. To create that expression requires that the user know how to compute the template app's total record count using fields available in the selection app.

For more information, see Building an on-demand app.

Using selection apps to generate on-demand apps does not require a user to understand the load script. Once an on-demand app navigation link has been created, a user can drag that navigation link onto the selection app's App navigation bar to create an app navigation point. On-demand apps are then generated from the navigation point.

Navigation points become available for on-demand app generation when the maximum row calculation from the expression in the navigation link comes within the required range. At that point, the user can generate an on-demand app. The user can also make another set of selections and generate additional apps based on those different selections.

For more information, see On-demand apps.

Navigation links have a limit on the number of on-demand apps that can be generated from the link. When the maximum number of apps has been generated, the user who is generating apps from the navigation point must delete one of the existing apps before generating a new on-demand app. The maximum number of generated apps applies to the on-demand app navigation link. If one on-demand app navigation point is created from the navigation link, then that navigation point would be able to create up to the maximum number. When multiple navigation points are created from the same navigation link, together those navigation points are limited to the maximum number set for the navigation link.

Navigation links also set a retention time for generated apps. On-demand apps are automatically deleted when their retention period expires.

Publishing on-demand apps

Most users will use on-demand and selection apps after they have been published. When selection apps are published to a stream, users who have proper permission on that stream can use them to make aggregate selections and generate on-demand apps from the navigation points included with the selection apps. As with all published apps, they cannot be changed after they have been published. To add navigation points, for example, the user would have to make a copy of the selection app.

In many cases, users only use generated on-demand apps. Each generated app can be published separately. In fact, the app navigation link can specify that apps generated from it be published to a specific stream automatically. Users would then explore the selected slices of data loaded with those generated on-demand apps on the stream to which the app was published.

Advantages of on-demand apps

On-demand apps help business users and IT departments derive value from big data environments in numerous ways. On-demand apps:

• Provide users with a "shopping list" experience that enables them to interactively populate their apps with a subset of data such as time period, customer segment, or geography.

• Provide full Qlik Sense functionality on a latent subset that is hosted in memory.

  In contrast, Direct Discovery, which can also manage large data sources, does not keep all relevant data in memory. With Direct Discovery, measure data resides at the source until execution.

  For more information, see Accessing large data sets with Direct Discovery.

• Enable IT to govern how large an app can be and invoke apps based on data volume or dimensional selections.

• Provide access to non-SQL data sources such as Teradata Aster, MapR, SAP BEx, and the PLACEHOLDER function in SAP HANA.

  Performing non-SQL queries is in contrast to Direct Discovery, which can only be used with SQL data sources.

• Allow customizable SQL and load script generation.
• Allow section access in all cases.