Skip to main content Skip to complementary content

Running a Job in Talend Administration Center

In the Job Conductor page of Talend Administration Center, you define an execution task to gather the script generation, deployment and execution phases of your MapReduce and Spark Batch Jobs.

Before you begin

  • The Databricks cluster to be used has been properly configured and is running.

  • You have created the use case Jobs described in the previous sections and run them successfully from Talend Studio.

Procedure

  1. Log in to Talend Administration Center with the account you have created in Setting up your first user and project.
  2. In the Menu tree view of your Talend Administration Center, click Job Conductor to display the Job conductor page.
  3. From the toolbar on the Job Conductor page, click Add > Normal Task to clear the Execution task configuration panel.
  4. In the Label field, enter the name you want to give to the task to be triggered. For example, getting_started.
  5. Click the icon to open a Job filter to search for the Job to be run from Job conductor and select it from the filter using its Latest version.

    For example, it can be the Spark Batch Job described in Joining movie and director information using an Apache Spark Batch Job.

    Once you have selected the Job, the Project, the Branch, the Name, the Version and the Context fields are all automatically filled with the related information of the selected Job.

  6. Select the Regenerate Job on change check box to regenerate the selected Job before task deployment and execution every time a modification is made to the Job itself.

    Note that if you selected Latest version, in case a new version of the Job is created in Talend Studio, the Job will be regenerated even if you did not select the Regenerate Job on change check box.

  7. Select the server on which the task should be deployed.

    In this scenario, the server is the Talend Runtime service you have configured in Connecting Talend Runtime Container to Talend Administration Center.

  8. Click Save to validate the configuration.

    This new task is added to the task list.

  9. In the Job conductor page, click the getting_started task to select it and on the toolbar, click Generate to allow the task to fetch the relevant Job script in the relevant project from the Talend Studio Repository and generates the code.

    Once done, the status of the task changes to Ready to deploy, meaning that the code generated is now ready to be deployed on the execution server.

  10. Click Deploy to deploy the Job on the execution server.

    Once done, the status changes to Ready to run. This means that the server has received the Job and is now ready to execute it.

  11. Click Run to execute the Job.

    Once done, the status switches back to Ready to run, which means that the Job can be run again if needed.

    In case the task did not complete properly, check the Error Status column as well as the task log for the Job completion information.

    Once done, you can check, for example in your Microsoft Azure Storage Explorer, that the output has been written in an ADLS storage system.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here