tKinesisOutput properties for Apache Spark Structured Streaming | Talend Components for Jobs Help
Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

tKinesisOutput properties for Apache Spark Structured Streaming

Last updated: 9/30/2026
Feature in preview

These properties are used to configure tKinesisOutput running in the Spark Structured Streaming Job framework.

The Spark Structured Streaming tKinesisOutput component belongs to the Messaging family.

The streaming version of this component is available in Talend Real-Time Big Data Platform and in Talend Data Fabric.

Basic settings

Properties Description
Schema and Edit schema

A schema is a row description. It defines the fields (columns) processed by the component. When you create a Spark Job, avoid the reserved word line when naming fields.

  • Built-In: create and store the schema locally for this component only.

  • Repository: reuse a schema created and stored in the Repository across projects and Job designs.

Click Edit schema to modify the schema. If you modify a Repository schema, the available options include:

  • View schema: view the schema without modifying it.

  • Change to built-in property: change the schema to Built-in for local changes.

  • Update repository connection: modify the Repository schema and choose whether to propagate the changes to other Jobs.
Configuration component Select the tKinesisConfiguration component to use for the Kinesis connection settings.
Stream name

Enter the name of the Kinesis stream you want to add data to.
Region Enter the AWS region of the Kinesis stream, for example "us-east-1".
Number of shards

Enter the number of partitions (shards in terms of Kinesis) to be created in the target Kinesis stream.
Checkpoint location

To use a cloud storage path, add the corresponding configuration component to the Job. For example, use tS3Configuration for Amazon S3, tGSConfiguration for Google Cloud Storage, or tAzureFSConfiguration for Azure Data Lake Storage.
Define partition key Select this check box to control how Kinesis partition keys are assigned. When selected, choose one of the following options:
  • Use dataset partition key: uses the partitionKey column from the input dataset as the partition key.
  • Use static partition key: uses a fixed literal string as the partition key for all records. Enter the value in the field that appears.

If cleared, a partition key is generated automatically based on the number of shards.
Set trigger Select this check box to display the Trigger interval field and configure the processing-time trigger interval. Only the processing-time trigger is supported. The default interval is 1 second.
Output mode The only supported output mode is Append, which adds new rows without modifying existing rows.

Advanced settings

Properties Description
Advanced options Add custom Kinesis connector options as key-value pairs by clicking the plus button.

Usage

Usage guidance Description
Usage rule

This component is used as an end component and requires an input link.

This component needs a Write component such as tWriteJSONField to define a serializedValue column in the input schema to send serialized data.

This component, along with the Spark Structured Streaming component Palette it belongs to, appears only when you are creating a Spark Structured Streaming Job.

Spark Connection

You need to use the Structured Streaming Configuration tab in the Run view to define the connection to a Spark cluster for the whole Job.

This connection is effective on a per-Job basis.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here