Skip to main content Skip to complementary content
Qlik Resources

Setting general connection properties

You can add a File endpoint to Qlik Replicate to use as a source. For information on how to add endpoints, see Defining and managing endpoints.

To add a File source endpoint to Qlik Replicate:

  1. In Tasks view, click Manage Endpoint Connections to open the Manage Endpoints Connections dialog box. Then click the New Endpoint Connection button.
  2. In the Name field, type a name for your endpoint. This can be any name that will help to identify the endpoint being used.
  3. In the Description field, type a description that helps to identify the File endpoint. This is optional.

  4. Select Source as the endpoint role.

  5. Select File as the endpoint Type.

    Information note

    When this endpoint is used as a duplicated source in a Log Stream Staging setup, select the Read changes from log stream check box and then select the relevant Log Stream Staging task from the drop-down list.

    For information on setting up and managing Log Stream Staging tasks, see Using the Log Stream.

  6. Configure the settings in the General tab as described in the table below.
General tab settings
Option Description

File Format

-
Field Delimiter

The delimiter used to separate columns in the source files. The default is a comma.

Example:

mike,male

Record delimiter

The delimiter used to separate records (rows) in the source files. The default is a carriage return (\n).

Example (Using an asterisk as the row delimiter)

mike,male*sara,female

Null value

The character used to indicate a null value in the source files.

Example (where * is the row delimiter and @ is the null value):

mike,male,295678*sara,female,@

Quote character

The character used at the beginning and end of a column that contains the column delimiter character. The default is the double-quote character ("). When a column that contains column delimiters is enclosed in double-quotes, the column delimiter characters are interpreted as actual data, and not as column delimiters.

Example (where a comma is the column delimiter):

"sunroof, power-steering"

Escape character

The character used to escape a string when both the string and the column containing the string are enclosed in quotation marks. Note that the string’s quotation marks will be removed unless they are escaped.

Example (where " is the quote character and \ is the escape character):

1955,"old, \"rare\", Chevrolet",$1000

Code page

Specify the code page of your source files if it is different from the default (65001).

Information note

Windows and Linux systems use different code page conventions. The specified code page must comply with the code page convention of the source file system.

Ignore records

Optionally, specify which header and footer rows in the source files to ignore. Make sure that the header and footer rows to ignore do not contain actual data.

Information note

The specified number will be ignored in Full Load tasks only.
Change Processing
Information note

Changes cannot be captured from Change Files that are present during the Full Load operation. Consequently, the Change Files should be placed in their source location(s) only after Full Load completes.

 -
Folder

Select this option if your Change Files contain the target table names and reside in a single folder. Then specify the folder location in the designated field. You can optionally use wildcard characters to only process files that match the specified pattern.

Example:

c:\temp\*changes.CSV

See also: Change files.

Use Reference Files

Select this option if you are using Reference Files to point to the location of the Change Files. Then specify one of the following:

  • The path to the Reference Files
  • The path to a specific Reference File
  • A path with a wildcard pattern that matches the Reference File names (e.g. C:\Reference Files\*.csv)

The folder can either contain a single reference file (which is continually appended with the Change File locations) or multiple reference files.

For information on when Reference Files should be used, see Reference files.

Change File path is preceded by table name

Select this option if each of the Change File paths in the reference files is preceded by a table name.

Information note

Selecting this option will disable the Table name check box in the Header columns ordinal position section.

Header column order

Specify the position of each header column in your Change Files. Apart from the data columns which must be positioned after the header columns, the columns can be positioned in any order.

So, for example, if your Change Files looked liked this:

DELETE,table1,timestamp,user1,dog,cat,bird

Then, the column positions would be set as follows:

Operations are in column: 1

Table names are in column: 2

Timestamps are in column: 3

User names are in column: 4

Start data from column: 5

For more information on the columns, see Change files.

See also Changing the ordinal positions of the user name and timestamp columns.

Start data from column:

Specify which column to start the actual data from. Note that data columns must be positioned after header columns. See the example in the description of the Header column order field above.

Changing the ordinal positions of the user name and timestamp columns

To change the ordinal positions of the "User name" and "Timestamp" columns, you first need to define two transformations (one for each column) that add the $AR_H_TIMESTAMP and $AR_H_USER header columns to the Change Files.

To add the columns to a single table (i.e. Change File):

  1. Open the Table Settings window as described in Defining transformations for a single table/view.

  2. Select the Transform tab and then click Add Column.

    A new row will be added to the Output table.

  3. Click the button at the end of the row (in the Expression column).

    This will open the Expression Builder Window.

  4. Select the column from the Header Columns tab and then click OK.

  5. Specify a name for the column. For example, if you selected the $AR_H_TIMESTAMP header column, specify "Timestamp".

  6. Repeat Steps 1-5 to add the other header column.
  7. Click OK to close the Table Settings window.

To add the header column to all tables (i.e. Change Files):

  1. Open the Global Transformation Rules window as described in Starting the Global Transformation Rules wizard.

  2. Click Next twice until you get to the How to transform screen.

  3. Click the Browse button next to the Computation expression field.

    This will open the Expression Builder Window.

  4. Select the column from the Header Columns tab and then click OK.
  5. Specify a name for the column. For example, if you selected the $AR_H_TIMESTAMP header column, specify "Timestamp".
  6. Click Finish.
  7. Repeat steps 1-6 to add the other header column.
Information note

To determine if you are connected to the endpoint you want to use or if the connection information you entered is correct, click Test Connection.

If the connection is successful a message in green is displayed. If the connection fails, an error message is displayed at the bottom of the dialog box.

To view the log entry if the connection fails, click View Log. The server log is displayed with the information for the connection failure. Note that this button is not available unless the test connection fails.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here