# Running tasks in parallel on Remote Engine

A Talend Remote Engine has always been able to run different tasks in parallel. From v2.12.0 onwards, it also supports concurrent run of identical tasks.

In other words, you can run multiple instances of a same task in parallel on a Remote Engine v2.12.0 and onwards.

This section explains the settings to be used to manage this parallel execution capability of a single Remote Engine, with or without this engine being part of a Remote Engine cluster.

## Before you begin

* Ensure that you have the latest engine version installed.
* Ensure to apply concurrent programming techniques on your Job design and implementation so that your task is thread-safe, that is to say, adapted to parallel executions, especially if you decide to enable parallel runs of a task on a single Remote Engine. For example, the parallel runs must have consistent access to shared resources.

## Procedure

1. Stop your engine.
2. Browse to the <RemoteEngineInstallationDirectory>/etc folder.
3. Open the org.talend.ipaas.rt.deployment.agent.cfg file and edit parameters as follows:

   The following table presents the editable parameters in org.talend.ipaas.rt.deployment.agent.cfg fileThe following table presents the editable parameters in org.talend.ipaas.rt.deployment.agent.cfgorg.talend.ipaas.rt.deployment.agent.cfg file
   | Parallel runs of identical tasks | <ul> <li>If you need to enable this support, ensure that the following                                    line is present:                                       <pre><code><pre><code><code>job.deployment.direct=true</code></code></pre></code></pre><p>For                                       Remote Engine v2.12.4 and onwards, direct deployment becomes                                       the default and only deployment used by the engine. This                                          <span>job.deployment.direct</span> parameter is thus                                       removed, since you do not need to set it anymore.</p><div>At                                       the same time, you must allow parallel executions of this                                       task to eventually make use of this engine configuration,                                       that is to say, perform either of the following operations:<ul> <li>In <span>Talend Management Console</span>, select the <span>Allow parallel execution of                                                this task</span> option when doing the task                                             configuration.</li> <li>Via API, set the                                                <span>parallelExecutionAllowed</span> parameter                                             as <kbd>true</kbd> when issuing requests to                                             create or update tasks.</li> </ul></div>Otherwise, parallel run does not apply despite this                                    engine configuration.</li> </ul> |
   | --- | --- |
   | Maximum number of concurrent runs | By default, only 3 parallel runs are allowed, whether the tasks to<br>be run are different or identical.<br><br><ul> <li>You can extend this limit to allow for as many parallel                                    executions of different tasks as you want, if you have enough                                    resources.<pre><code><pre><code><code>max.deployed.flows=3</code></code></pre></code></pre>If you set                                    the limit of concurrent tasks to <kbd>0</kbd>, there                                    will be no limitation on the number of tasks you can run on the                                    remote engine in parallel, provided you have enough resources.                                 </li> </ul> |
   | In a Remote Engine cluster | When a Remote Engine has been added to a cluster, parallel runs of<br>identical or different tasks are by default spread out across the<br>cluster.<br><br>However, when the tasks to be run are identical, you can<br>boost the parallel executions on a single Remote Engine, by setting<br>the following parameter to be<br>true:<br><br>```<br>cluster.engine.same.task.parallel.run=true<br>``` |
4. Restart your engine.

## Results

You can start executing tasks on your Talend Remote Engine after it becomes Available again in Talend Management Console.

Troubleshooting: When multiple threads of your task are being deployed, one thread may accidentally overwrite another thread, resulting in the failure of the overall deployment.

To repair this issue, do the following:

1. Open the org.talend.ipaas.rt.deployment.agent.cfg file.
2. Add a new parameter to the file:
   ```
   lock.same.task.jobs.parallel.deployment=true
   ```
3. Save the file and restart your engine.

This parameter locks a thread during the deployment to prevent it from being overwritten by the other parallel threads of the same Job.