Upgrading Remote Engine v2.11.8 onwards for data service Job or Route tasks
Upgrade your Remote Engine without the need to undeploy or redeploy any data
service Job or Route tasks currently running on the engine.
Microservice tasks are automatically halted with the first Remote Engine shutdown and
resumed upon the startup of the new Remote Engine version.
Before you begin
The Remote Engine to upgrade must be v2.11.8 and onwards. If you are using
v2.11.7 or an earlier version, upgrade them to the latest version before
starting this procedure as explained in Upgrading Talend Remote Engine.
Generate a service account token (SAT) to use a service account to perform
this operation. For more information on how to generate this token, see Generating a service account
token.
Using a service account is a widely accepted security
practice for automation in a production environment. While using a personal
access token (PAT) is technically possible, it is recommended to utilize a
service account for enhanced security.
Gather the IDs of the engines to be upgraded, using either the related API
endpoint or the Talend Management Console web page:
API endpoint: Get available Remote Engines.
In the response, the engine IDs are found in the id field. Copy
the IDs of the engines to be upgraded and keep them in a place of your
choice for later use.
Talend Management Console web page: a Remote Engine ID is found in the
Overview tab of the Engine
details blade.
If the engine in question is in a cluster, this URL reads
like
Use the specific API endpoint or the Talend Management Console web UI to unpair the Remote Engine to be upgraded from the designated engine
logical entity on Talend Management Console.
$RE_ENGINE_ID is for the ID of the engine
to be unpaired.
$RE_TOKEN is the service account token,
which is recommended, or the personal access token of the
account to be used.
You can either replace these variables with their actual values
in your command, or define them in the environment of your API
requests. If you use Talend API Tester to issue your API endpoint, see Creating an environment to
learn about how to define variables.
When the command is
executed, a new preauthorized key is automatically generated. Keep
this key in a place of your choice, because you need to use it to
pair the new Remote Engine in the following steps.
Talend Management Console web page: follow this procedure to find the
Reset the connection section in the
Engine details blade of the engine to be
unpaired, and from that section, click Reset the
key to unpair the engine and get a new pairing key.
Wait for at least 2 minutes to ensure that the engine is correctly
unpaired.
Example
The following message from the engine logs is an example that indicates the
unpairing has been correctly completed.
On the host machine of the unpaired engine, stop the engine using the following
command:
<RemoteEngine_installation>/bin/stop
If this engine
has been installed as a system service, follow your OS procedure to shut down
the service.
Install and configure the new Remote Engine version.
If you use the installer to install this new engine, do not use the
actual preauthorized key yet when you are prompted to provide a key;
provide a fake authorized key instead.
If you have installed the new engine as a system service, stop the
service before proceeding to the following steps.
Copy data from the old engine to the new engine.
This copy helps ensure the continuous running of your data service Job tasks
or Route tasks. For this reason, copy each type of data to their designated
folders as expected by Remote Engine.
Copy-paste the dsrunner folder from
Old_RemoteEngine_installation/data/ to
New_RemoteEngine_installation/data/.
This folder contains the task information.
Copy-paste the cls folder from
Old_RemoteEngine_installation/data/ to
New_RemoteEngine_installation/data/.
This folder contains the task logs.
On the new Remote Engine, update the path to the new engine to be used
in every .metadata file that is stored in each of
the task folders in
New_RemoteEngine_installation/data/dsrunner/exec/osgi.
Example
For example, in this folder
New_RemoteEngine_installation/data/dsrunner/osgi/02141bff-bffe-4975-8835-7dd9ff77b62a,
open the .metadata file and find the following
line, which is the path to the engine to be
used:
Still on the new Remote Engine, update the path to the log files for
each execution.
This information is found in every
<TASKID>-<EXECUTION_ID>-state.dat file
that is stored in
New_RemoteEngine_installation/data/cls.
Example
For example, open the
New_RemoteEngine_installation/data/cls/640ef0c6a45e743a0b2580ca-5f69d7c4-e3d8-4d41-b248-b94a20d7cbf0-state.dat
file and find the following line, which is the path you need to
update:
Pair the new Remote Engine to the same Talend Management Console engine item that you previously unpaired from the old Remote Engine.
In the
New_RemoteEngine_installation/etc/org.talend.ipaas.rt.pairing.client.cfg
file, specify the endpoint for pairing of Talend Management Console in the following line:
If you have installed the new engine using the installer, you can
skip this step because the endpoint should have been automatically
specified.
In
New_RemoteEngine_installation/etc/preauthorized.key.cfg,
replace the fake preauthorized key with the actual one generated by the
unpairing call at the beginning of the procedure.
If you have lost this preauthorized key, find it back in the
Remote Engine key field of the engine item to
be paired on Talend Management Console. For further information about how to find this key, see Creating a Remote
Engine.
Verify the task status on Talend Management Console
by checking that data service Job tasks and Route tasks are up and running. Logs and
metrics should also be available for all these tasks.
Did this page help you?
If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!
If the engine in question is in a cluster, this URL reads likeIn this case, the ID is 646c790a2373b74412c26047.
Then replace it with the path to the new Remote Engine: