Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Enabling runtime lineage for Jobs

Talend Studio allows you to enable runtime lineage for standard Jobs to support analysis in Talend Data Catalog.

When you execute a standard Job with runtime lineage enabled, the system writes metadata (Job name, component name, schema, query) to a JSON file.

Information noteNote: To fully use this feature, you must install Talend Data Catalog.
Information noteNote: If you have a Qlik Talend Cloud license, you can also generate lineage in Qlik Cloud. See Publishing datasets and lineage to Qlik Cloud.

Procedure

  1. Go to the Talend Studio installation directory.
  2. Add the -Dtalend.lineage.enabled=true attribute in the corresponding .ini file according to your operating system to enable the runtime lineage feature in Talend Studio.
    You can also clear the Disable Lineage checkbox in the Job Settings view of the Project Settings dialog box to enable runtime lineage for the Job.
    After you apply this setting, all built Jobs automatically include the code for lineage.
  3. Save the file and start Talend Studio.
  4. Add the JVM parameter -Druntime.lineage.outputpath=/Users/a_path to set the output path for the Job using one of the following methods:
    Each time you execute a standard Job with runtime lineage enabled, the system saves a JSON file in this format: <output_path>/<project>/<jobname>/<version>/runtime_log_<timestamp>.json, where
    • <output_path> is the path specified in the output path field.
    • <project> is the project name.
    • <jobname> is the Job name.
    • <version> is the Job version.
    • <timestamp> is the timestamp when the system generates the JSON file.
    Information noteNote: You must specify the output path to save the JSON files. If you specify the output path in multiple places, the system uses the following precedence:
    1. JVM parameter for a specific Job.
    2. Output path field.
    3. JVM parameter for all Jobs.
    4. JVM parameter in the shell script.

What to do next

For information about importing runtime lineage data into Talend Data Catalog, see Talend Import Bridge.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here