Creating a Spark Job | Talend Studio Help
Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Creating a Spark Job

Last updated: 9/30/2026
Create a Big Data Batch or Big Data Streaming Spark Job from the Job Designs node in the Repository tree view.

Procedure

  1. Right-click the Job Designs node and in the contextual menu, select Create Big Data Batch Job or Create Big Data Streaming Job.
    Then the corresponding New Big Data Batch Job or New Big Data Streaming Job wizard appears.
  2. From the Framework drop-down list, select Spark for a Batch Job, or
    Feature in preview
    Spark Structured Streaming     or
    Availability-noteDeprecated
    Spark Streaming     for a Streaming Job.
  3. In the Name, the Purpose and the Description fields, enter the descriptive information accordingly. Among the information, the Job name is mandatory.
    Once done, the Finish button is activated.
  4. If you need to change the Job version, click the M and the m buttons next to the Version field to make the changes.
    If you need to change the Job status, select it from the drop-down list of the Status field.
    If you need to edit the information in the uneditable fields, select File > Edit Project properties from the menu bar to open the Project Settings dialog box to make the desired changes.
  5. Click Finish to close the wizard and validate the changes.
    Then an empty Job is opened in the Talend Studio workspace and the available components for Spark appear in the Palette.

Results

In the Repository tree view, this created Spark Job appears automatically under the Big Data Batch node under Job Designs.

Add components from the Palette to the workspace, then connect and configure them to design the Job.

To configure the runtime environment for your Spark Job, define the connection in the Spark configuration tab of the Run view. For more information, see Defining Spark Batch connection details.

Feature in preview
For a Spark Structured Streaming Job, see Defining Spark Structured Streaming connection details.

You can also create these types of Jobs by writing their Job scripts in the Jobscript view and then generate the Jobs accordingly. For more information on using Job scripts, see Talend Job scripts reference guide.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here