Creating Job tasks

Job tasks are executable integration processes that contain a Job artifact published from Talend Studio.

If contexts have been defined for this Job artifact in Talend Studio, the default context is automatically published with this Job artifact. For recommended rules about defining these contexts in Talend Studio, see Using contexts and variables.

The values in this context configuration are cleared in the exported files for security reasons. You need to enter the context values again in Talend Management Console.

Before you begin

You must have the Author permission on the workspace where the task is located to create or edit a Job task and manage its configuration.
The Author permission allows you to configure the Artifact and Parameters steps when creating or editing a task.
You must have the Execute permission on the workspace where the task is located to manage the run configuration and schedule, such as setting engine, log level or execution timeout.
The Execute permission allows you to configure the Processing and Schedule steps when creating or editing a task.

Procedure

Open the Management page.
Go to the Tasks tab.
Select the environment and the workspace using the faceted search at the top of the page.
Click Add task.
In the Artifact step, select the Job artifact type.
Select the artifact you want to execute from the drop-down list.
The details and description of the artifact appears.
Select the version of the artifact to be used in the task.

You can select Always use the latest available artifact version if you want your task to be updated automatically each time a new version of the artifact is published. You can also select a specific version and update it manually as needed.

Information noteNote: This option overrides the Update corresponding job task option in Talend Studio.

The details and description of the artifact are refreshed automatically.
Optional: Select the Override parameter values with artifact defaults check box.
This option is only available if you selected Always use the latest available artifact version in the Artifact version field. If you select it, the parameters defined in the task are replaced with the default artifact parameters each time a new version of the artifact is published.
Select the workspace for the task.
Give a name to the new task.
By default, the task name is the name of the artifact from which it is created.
Information noteNote: Tasks must have a unique name within a workspace.
Optional: Add a tag to the task.
1. Select an existing tag from the drop-down list.
2. Create a new tag by typing in the name and clicking on it in the drop-down list.
Click Continue.
In the Parameters step, enter the value for the parameters of the Job.
You can see the description of each parameter under its field.

In this step, when the Show order as in Studio toggle is disabled, you can only set the values to parameters you have defined with the parameter_ prefix in Talend Studio.

Information noteNote: Use \\ instead of \ when the value could be interpreted as a special character, for example, \t.

When being displayed in Talend Management Console, the first letters of each word in the parameter names are automatically changed to upper case and underscores (_) to whitespace, for example, city_name becomes City Name.

This is for display purpose only, that is to say, when you need to use a parameter name in your programs, for example, an API request, do not use the display name but use the name from Talend Studio.
Optional: In the Advanced parameters section, enter the values for the parameters you have defined in Talend Studio without prefixes, and connection_ parameters that are not defined in the Job. This section is available when the Show order as in Studio toggle is disabled. Otherwise, all parameters are in the Parameters step.

Information noteNote: Use \\ instead of \ when the value could be interpreted as a special character, for example, \t.
Optional: In the Connections step, select the appropriate connection from the drop-down list.
You can also create a new connection based on the parameters already defined in the artifact.
Optional: In the Resources step, select the appropriate resource to use in the task.
In the Processing step, select the engine or engine cluster on which you want to run your task from the Engine drop-down list.
- Cloud
- Cloud exclusive
- Remote Engine
- Remote Engine cluster
Optional: Select a run profile.
You can choose a previously created run profile for the selected Remote Engine or cluster to set the JVM parameters for the task run.

If you are using a Remote Engine or cluster, this feature is supported from Remote Engine 2.9.1 onwards.
Set the log level. Five levels can be included in your logs, but not all are available as standalone options.
- Three log levels are displayed on the drop-down list for immediate selection: Info, Warning, and Error.
- The Debug level, not available on the list, is automatically used when you manually start a failed task as a debug run. For further information, see Debugging Job tasks.
- The Fatal level is not available as a standalone option but included in any of the other log levels.
- Level overlapping.
  - Debug is the lowest level.
    In the debug run, all logs are included (from lowest to highest): Debug, Info, Warning, Error, and Fatal.
  - Info includes logs for Info, Warning, Error, and Fatal, but excludes logs for Debug.
  - Error is the highest of the three levels available for immediate selection. This level includes logs for Error and Fatal only.
    By default, Fatal errors are displayed as Error logs. To make it explicitly displayed as Fatal logs in Talend Management Console, ensure that the job.log.fatal.level parameter is present in the <RemoteEngineInstallationDirectory>/etc/org.talend.ipaas.rt.jobserver.client.cfg file and its value is true.
- Recommended log level for a production environment.
  To avoid overwhelming the log generation process, which potentially leads to data loss and process delays, set the log level to Warning for Jobs deployed to production, which includes Warning, Error, and Fatal.
- If you select Off, Job execution logs are not generated at all on the engine side. Only the summary logs of a task run are generated and sent to Talend Management Console. These logs provide a summary of the task run, such as when it started and finished and its success or failure status.
  Information noteTip: Some standard output (stdout) and standard error (stderr) information can still be written in a stdOutErr_.log file in the JobServer directory on your engine, but this is not sent to Talend Management Console.
Optional: Select the Allow parallel run of this task option so that the task can be run in parallel when using a Cloud Engine, Cloud exclusive, Remote Engine, or cluster.

If you are using a Remote Engine to run identical tasks simultaneously, this feature is supported from Remote Engine 2.12.0 onwards.

By default, you can run three different or identical tasks in parallel on a single Remote Engine. For further information about the related configuration, see Running tasks in parallel on a single Remote Engine.

Before you disable this option, stop the task or wait for the end of the run. Otherwise, the parallel run is still allowed.
Optional: If you have selected a Remote Engine or cluster, you can run your Job as another user by entering their username in the Run as impersonated user field.
This feature is supported only if the Remote Engine is installed on Unix or similar environments.

For more information about user impersonation, see Configuring user impersonation for Talend Remote Engine.
Optional: Set the task execution timeout by selecting Custom then entering a value.

The default value is 5 minutes and the minimum is 1.
Optional: In the Schedule step, select the execution frequency and if necessary the trigger timeout.

You can add one or more triggers to schedule your task or you can manually start the task without a trigger.

By default, the trigger timeout value is the same as the task timeout.
Trigger timeout overrides task timeout.

For more information, see Scheduling Job tasks.
Click Save and close.

Results

Your task is ready to run on Talend Management Console.

Note:

Exponential back-off retry pattern has been adopted to provide not only faster first and second retries (1 minute and 3 minutes respectively after the initially failed attempt), but also better global behaviour on peak workload situation.

For example, when you try to execute a task that is already running, the execution fails. Talend Management Console retries the execution with incrementally increased intervals, that is to say, the first retry takes place at the end of 1 minute only, the second one 2 minutes, then subsequently 4 minutes and 8 minutes.

If the fourth retry fails, you will get the error message Exceeded the limit of deployment attempts: another instance of the same task was running on the engine at same time. Try to run the task later.

Troubleshooting: If the task execution schedule is too frequent, it may result in the parallel run of the same task on two different Cloud Engines or Remote Engine cluster. To avoid this issue, run your task in an environment with only a single Cloud Engine allocated to it or schedule the task on a Remote Engine.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here