JSON: New source

Adding a new JSON entity to a new source

Select radio button: To new Source

This will kick off the Add Data wizard to create a new source and define a new entity within the source.

Source Connection

Choose Connection: Choose a source connection that will connect to an external JSON source on the required protocol (JSON ingest is supported for ADLS, FTP, HDFS, KAFKA, LOCALFILE, S3, SFTP, WASB, WASBS) This example uses JSON source type connection to LOCALFILE.

Add New Source Name: Configurable. This is source that the new entity will be added to.

Default Entity Level: Specify level of data management (see System Settings: Level Control)

Select Source Hierarchy: Configurable. Choose from dropdown.

Inbound Protocol: Pre-defined Inbound Protocol auto-populates for the selected source connection

Base Directory: Configurable. This is the directory where data will be stored in file system. [This value is specified in core_env property: localfile.base.dir.source.connection. This property limits access and browsing to subdirectories of the specified location.]

Groups: Select Group(s) requiring access from the dropdown options. At least one group must be added for the data to be discoverable.

Click Next.

File Format

File Path:  Enter the file path to the originating external source, ending with the file name and file format suffix. Select file browser to explore local server directory for filepath to data.

Data Format Specification: JSON File Path is required. Users are then required to upload a JSON .hen definition file to validate and enforce the schema of the JSON file. This is a hand-written text file.

See JSON .hen specification: Creating hierarchical extraction and normalization files

Make sure to view the JSON definition file. This step will validate verify the .hen file. 

Select Next

Select Entities & Fields

The entities specified within the .hen validation file display; this step allows the user to select the entities to generate (with ability to rename).

Select Fields Metadata Preview.

NOTE that JSON fields can be selected and renamed but not added or removed.

Internal file formatType

Options include:

• AVRO
• ORC
• ORC_ALL_STRING
• PARQUET
• PARQUET_ALL_STRING
• TEXT_TAB_DELIMITED

Qlik Catalog Stored Format Type step is skipped for single node environments where format type is always TEXT_TAB_DELIMITED.

Once the Entity has been saved, navigate to Entities screen in Source to load the data.

Select Load from the More dropdown.