Setting general properties

This section describes how to configure general properties. For an explanation of how to configure advanced properties, see Setting Advanced Properties below.

To configure general properties for the File target endpoint:

1. In Tasks view, click Manage Endpoint Connections to open the Manage Endpoints Connections dialog box. Then click the New Endpoint Connection button. For more information on adding an endpoint to Qlik Replicate, see Defining and managing endpoints.
2. In the Name field, type a name for your endpoint. This can be any name that will help to identify the endpoint being used.
3. In the Description field, type a description that helps to identify the File endpoint. This is optional.
4. Select Target as the endpoint role.
5. Select File as the endpoint Type.
6. In the Target folder field, specify the full path of the folder to which you the target files to be written.
7. Configure the remaining settings in the General tab as described below.

File Format

Format

You can choose to create the target files in CSV or JSON format.

In a JSON file, each record is represented by a single line, as in the following example:

About delimiters

Delimiters can be standard characters or a hexadecimal (hex) value. Note that the "0x" prefix must be used to denote a hexadecimal delimiter (e.g. 0x01 = SOH). In the Field delimiter, Record delimiter and Null value fields, the delimiter can consist of concatenated hex values (e.g. 0x0102 = SOHSTX), whereas in the Quote character and Escape character fields, it can only be a single hex value.

Field delimiter

The delimiter that will be used to separate fields (columns) in the target files. The default is a comma.

Example using a comma as a delimiter:

"mike","male"

Record delimiter

The delimiter that will be used to separate records (rows) in the target files. The default is a newline (\n).

Example:

"mike","male"\n

"sara","female"\n

Null value

The string that will be used to indicate a null value in the target files.

Example (where \n is the record delimiter and @ is the null value):

"mike","male",295678\n

"sara","female",@\n

Quote Character

The character that will be used at the beginning and end of a text column. The default is the double-quote character ("). When a column that contains column delimiters is enclosed in double-quotes, the column delimiter characters are interpreted as actual data, and not as column delimiters.

Example (where a @ is the quote character):

@mike@,@male@

Escape character

The character used to escape a quote character in the actual data. The default is the double-quote character (").

Example (where " is the quote character and \ is the escape character):

1955,"old, \"rare\", Chevrolet","$1000"

Add metadata header

When the target storage format is set to Text, you can optionally add a header row to the data files. The header row can contain the source column names and/or the intermediate (i.e. Replicate) data types.

Example of a target file with a header row when both With column names and With data types are selected:

Position:DECIMAL(38,0),Color:VARCHAR(10)

1,"BLUE"

2,"BROWN"

3,"RED"

...

Code page

Specify the code page of your target files if it is different from the default (65001).

File Attributes

• Maximum file size - The maximum size a file can reach before it is closed (and optionally compressed). This value applies both to data files and to Reference Files.

  For information on generating reference files, see Generating reference files.

• Compress files using - Choose GZIP to compress the target files or NONE (the default) to leave them uncompressed.

Change Processing

• Consider state idle when no changes have been processed for - Specify how long to wait before considering the state to be idle. In idle state, you can apply changes to files using data that has already been processed if the specified size and time conditions are met (see below).
• Apply/store changes when:
  • File size reaches - Specify the maximum size of the data required in order to apply changes to the target file in idle state.
  • Elapsed time reaches - Specify the maximum time to wait before applying the changes in idle state.

• Allow a single transaction to be split into multiple files - By default, a single transaction will not be split across multiple files, regardless of the values specified in the File size reaches and Elapsed time reaches fields. This is important for organizations who require files to contain transactions in their entirety. However, this may also result in very large file sizes. For example, if the File size reaches value is 32 MB and Replicate starts to apply changes for a new 2 GB transaction at 31 MB, the target file will only be closed at 2.031 GB.

  You should therefore select this option if it is critical that the values in the File size reaches and Elapsed time reaches fields are adhered to (even if it means splitting a transaction across multiple files).

Metadata Files

Create metadata files in the target folder - When this option is selected, for each data file, a matching metadata file with a .dfm extension will be created under the specified target folder. The metadata files (which are in standard JSON format) provide additional information about the task/data such as the source endpoint type, the source table name, the number of records in the data file, and so on.

For a full description of the metadata file as well as possible uses, see Metadata file description .