Understanding the Talend JobServer clean-up cycle
The Talend JobServer cleans up Job artifacts on a schedule, based on data cleaning parameters.
You can customize the Talend JobServer clean-up schedule by changing the parameters in the TalendJobserver.properties file, listed under Temporary data cleaning parameters. Configuring these values is optional.
General clean-up frequency
The general clean-up schedule is defined by the FREQUENCY_CLEAN_ACTION parameter. You can disable the general clean-up by setting this parameter to 0.
Job repository and archive clean-up
- The Job is not running.
- Either the MAX_DURATION_BEFORE_CLEANING_OLD_JOBS or MAX_OLD_JOBS parameter is met.
Job execution log clean-up
Job logs are cleaned up in the next cycle when the following conditions are met:
- Either the MAX_DURATION_BEFORE_CLEANING_OLD_EXECUTIONS_LOGS or MAX_OLD_EXECUTIONS_LOGS parameter is met.
- The Job execution is released. A Job is released after 50 Job executions. You can change the Job release frequency by changing the MIN_NUMBER_JOB_EXECUTIONS_BEFORE_RELEASE parameter.
A log without errors is released if the elapsed time since the start of the log is greater than the time defined in the MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_NORMAL_CASE and MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_ABNORMAL_CASE parameters.
For each MAX_OLD_* and MAX_DURATION_* parameter pair, whichever is reached first will trigger a cleaning action.
Talend JobServer clean-up parameters
| Parameter | Default | Description |
|---|---|---|
| FREQUENCY_CLEAN_ACTION | 10 minutes | Defines the time between each Talend JobServer cleaning action. Set this value to 0 to disable automatic cleaning. |
| MAX_OLD_JOBS | 200 | Defines the maximum number of artifacts and deployed Jobs to keep. Once this value is passed, artifacts are deleted beginning with the oldest. Set this value to 0 to disable this parameter. |
| MAX_DURATION_BEFORE_CLEANING_OLD_JOBS | 3 months | Defines the maximum before cleaning archives and deployed Jobs. Set this value to 0 to disable this parameter. |
| MAX_OLD_EXECUTIONS_LOGS | 1000 | Defines the maximum number of execution logs to keep. Once this value is passed, logs are deleted beginning with the oldest. Set this value to 0 to disable this parameter. |
| MAX_DURATION_BEFORE_CLEANING_OLD_EXECUTIONS_LOGS | 3 months | Defines the maximum time before cleaning execution logs. Set this value to 0 to disable this parameter. |
| MIN_NUMBER_JOB_EXECUTIONS_BEFORE_RELEASE | 50 | Defines the minimum number of Job executions before a Job is released. Job logs cannot be cleaned until the Job is released. |
| MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_NORMAL_CASE | 5 minutes | Defines the maximum time for a normal execution before a Job is released. |
| MAX_DURATION_BEFORE_JOB_EXECUTION_RELEASE_ABNORMAL_CASE | 24 hours | Defines the maximum time before an abnormal execution before a Job is released. The abnormal duration applies to Job execution log files that contain errors. |