Skip to main content Skip to complementary content

Mainframe: New source

Mainframe: File Load Preparation: Multiple Record Layout Identifier (01 level)

Adding a new Mainframe entity to a new source

Select radio button: To new Source

This will kick off the Add Data wizard to create a new source and define a new entity within the source.

Source wizard select add data to new source

Source Connection

Choose Connection: Choose a source connection that will connect to an external Mainframe source on the required protocol (Mainframe ingest is supported for ADLS, FTP, HDFS, LOCALFILE, OPENCONNECTOR, S3, SFTP, WASB, WASBS) This example uses JSON source type connection to LOCALFILE.

Add New Source Name: Configurable. This is the source that the new entity will be added to.

Default Entity Level: Specify level of data management

(see System Settings: Level Control)

Select Source Hierarchy: Configurable. Choose from dropdown.

Inbound Protocol: Pre-defined Inbound Protocol auto-populates for the selected source connection

Base Directory: Configurable. This is the directory where data will be stored in file system. [This value is specified in core_env property: localfile.base.dir.source.connection This property limits access and browsing to subdirectories of the specified location.]

Groups: Select Group(s) requiring access from the dropdown options. At least one group must be added for the data to be discoverable.

Click Next.

Complete source connection fields

Base Directory Folder Browser for discovery of directories (Select Folder icon)

Browse base directory subfolders

File Format

File Path: Enter the exact file path to the (originating source) entity, ending with the exact file name and file format suffix. Select the file browser to explore the local server directory for the filepath to data.

Copybook:

Select and upload a copybook from local desktop machine. Click on Next.

Copybooks are text documents loaded in tandem with Mainframe files that validate and enforce record and field structure. Copybook file properties include data field definitions, metadata, and procedure codes for Mainframe data files.

The Copybook used in this example contains multiple record types at the 01 level. Separate entities are created from these specifications.

Copybook example text:
01 HEADER. 20 RECORD-TYPE PIC X(4). 88 HEADER-RECORD-FLAG VALUE 'HEAD'. 20 PROCESSING-SYSTEM PIC X(4). 20 PROCESSING-DATE PIC X(8). 20 FILLER PIC X(48). 01 TRAILER. 20 RECORD-TYPE PIC X(4). 88 TRAILER-RECORD-FLAG VALUE 'TRLR'. 20 RECORD-COUNT PIC 9(4). 20 FILLER PIC X(56). 01 LOAN. 20 RECORD-TYPE PIC X(4). 88 LOAN-RECORD-FLAG VALUE 'LOAN'. 20 LOAN-TYPE PIC X(4). 88 AUTO-LOAN-FLAG VALUE 'AUTO'. 88 HOME-LOAN-FLAG VALUE 'HOME'. 88 EDUC-LOAN-FLAG VALUE 'EDUC'. 20 LOAN-NUMBER PIC 9(4). 20 CUSTOMER-ID PIC X(4). 20 LOAN-AMOUNT PIC 9(6)V9(2). 20 LOAN-DETAIL-AREA PIC X(40). 20 AUTO-LOAN REDEFINES LOAN-DETAIL-AREA. 30 AUTO-TYPE PIC X(8). 30 AUTO-MODEL-YEAR PIC 9(4). 30 FILLER PIC X(28).

Copybook example screenshot

Example of copybook with multiple record types at the 01 level

Provide filepath to data and upload copybook from local machine

Local File Browser for discovery of Local Server directories (Select Folder icon)

Browse directories for data file

Select Entities & Fields

Select entities to load data to. Rename entities as required due to naming conflict of existing entities. Fields display by expanding the icon caret expand collapse caret to the left of the entity name (field name selections are not editable as in other source types)Select entities to add to the source.

Stored Format Type

Options include:

  • AVRO
  • ORC
  • ORC_ALL_STRING
  • PARQUET
  • PARQUET_ALL_STRING
  • TEXT_TAB_DELIMITED

Qlik Catalog Stored Format Type step is skipped for Single Server environments where Stored Format Type is always TEXT_TAB_DELIMITED.

Specify stored file format

Load Data to the Entity(s). Once the Entity has been saved, navigate to Entities screen in Source to load the data.

Select Load from the More dropdown.

Load data to the entity

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!