Defining Local connection parameters with Spark Structured Streaming | Talend Studio Help
Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Defining Local connection parameters with Spark Structured Streaming

Last updated: 10/1/2026
Preview

About this task

Talend Studio connects to your local file system on your machine to run the Job. Talend Studio builds the Spark environment in itself at runtime.

Procedure

  1. Click the Run view beneath the design workspace, then click the Structured Streaming configuration view.
  2. Select Built-in from the Property type drop-down list.
    If you have already set up the connection parameters in the Repository as explained in Centralizing a Hadoop connection, you can easily reuse it. To do this, select Repository from the Property type drop-down list, then click […] button to open the Repository Content dialog box and select the Hadoop connection to be used.
    Information noteTip: Setting up the connection in the Repository allows you to avoid configuring that connection each time you need it in the Structured Streaming configuration view of your Jobs. The fields are automatically filled.
  3. Select Universal from the Distribution drop-down list, the Spark version from the Version drop-down list, and Local from the Runtime mode/environment drop-down list.
  4. Select the Define a streaming timeout (ms) check box and, in the field that is displayed, enter the time frame at the end of which the Spark Structured Streaming Job automatically stops running.
  5. If you run your Spark Job on Windows, specify the location of the winutils.exe program:
    • If you want to use your own winutils.exe file, select the Define the Hadoop home directory check box and enter its folder path.
    • Otherwise, leave the Define the Hadoop home directory check box clear. Talend Studio will generate and use a directory automatically for this Job.
  6. In the Spark directory field, enter the local path where Talend Studio stores temporary files, like JARs to transfer.
    If you run the Job on Windows, the default disk is C:. Leaving /tmp in this field will use C:/tmp as the directory.
  7. Select the Set tuning properties check box to define the tuning parameters for your Spark Structured Streaming Job.
    Information noteImportant: You must define the tuning parameters otherwise you can get an error (400 - Bad request).
  8. In the Advanced properties table, add any Spark properties you want to override the defaults set by Talend Studio.

Results

The connection details are complete.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here