Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Configuring lineage history retention

Last updated: 9/9/2026
By default, Talend Remote Engine keeps lineage data on disk for 7 days after each task run. Adjust the retention period and cleanup frequency to match your disk management requirements.

Before you begin

Enable lineage collection for your Job task in Talend Management Console. See Enabling lineage collection for Job tasks.

About this task

Talend Remote Engine processes the lineage data produced by data integration Jobs. By default, lineage history is enabled. The engine moves each processed execution folder to a history sub-folder inside the lineage data directory instead of deleting the folder. A cleanup task runs periodically to delete entries older than the configured retention period.

Lineage data is first generated in the following directory: <RemoteEngineInstallationDirectory>/data/lineage/

Procedure

  1. Stop the Talend Remote Engine if it is running.
  2. Open the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.engine.metrics.storage.cfg file.
  3. Add any properties whose default values you want to change.

    By default, the file is empty and Talend Remote Engine applies its default lineage history settings. Add only the properties you need.

    lineage.history.enabled=true
lineage.history.retention.time.period.minutes=10080
lineage.history.cleanup.time.interval.minutes=150
    Property Default Description
    lineage.history.enabled true When true, moves the processed execution folder to history instead of deleting it. Set to false to delete processed execution folders immediately.
    lineage.history.retention.time.period.minutes 10080 (7 days) Number of minutes to keep a history entry before automatic deletion. The age is calculated from the timestamp in the folder name (yyyyMMdd-HHmmss).
    lineage.history.cleanup.time.interval.minutes 150 (2 hours 30 minutes) How often the cleanup task runs, in minutes.
  4. Save the file.
  5. Restart your Talend Remote Engine.

Results

The Talend Remote Engine retains processed execution data under <RemoteEngineInstallationDirectory>/data/lineage/history/ for the configured duration, then deletes it automatically.

Information noteTip:

When investigating lineage issues, Qlik Support can ask you to provide the contents of the history folder for diagnosis. When lineage generation fails, the raw files related to that task run are sent to the history folder.

If a send attempt fails, the execution data stays in the lineage folder and the engine retries automatically.

What to do next

To learn more about the complete lineage workflow, from data generation to lineage analysis, see Publishing datasets and lineage to Qlik Cloud.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here