# tKinesisInput properties for Apache Spark Streaming (deprecated)

Availability-note[Deprecated](https://help.talend.com/r/en-US/Cloud/labels-on-talend-help-center/availability-deprecated)

This content is deprecated.

These properties are used to configure tKinesisInput running in the Spark Streaming Job framework.

Information noteDeprecation: Spark Streaming is deprecated beginning with the R2026-09 release. Use Spark Structured Streaming instead.

The Spark Streaming tKinesisInput component belongs to the Messaging family.

The streaming version of this component is available in Talend Real-Time Big Data Platform and in Talend Data Fabric.

## Basic settings

| Properties | Description |
| --- | --- |
| Schema and Edit<br>Schema | A schema is a row description. It defines the number of fields<br>(columns) to be processed and passed on to the next component. When you create a Spark<br>Job, avoid the reserved word line when naming the<br>fields.<br><br>The schema of this component is read-only. You can click<br>Edit schema to view the schema.<br><br>This read-only *payload* column is used to carry the body<br>of the Kinesis message to be processed.<br><br>The input message body can use very different data formats. For example, if its format is<br>JSON, you need to use tExtractJSONField following tKinesisInput to extract the data to be processed from this<br>body. |
| Access key | Enter the access<br>key ID that uniquely identifies an AWS Account. For further information about how to<br>get your Access Key and Secret Key, see [Getting Your AWS Access<br>Keys](http://docs.aws.amazon.com/ses/latest/DeveloperGuide/get-aws-keys.html). |
| Secret key | Enter the secret access key,<br>constituting the security credentials in combination with the access Key.<br><br>To enter the password, click the [...] button<br>next to the password field, enter the password in double quotes in the pop-up dialog box,<br>and click OK to save the settings. |
| Stream name | Enter the name of the Kinesis stream you want tKinesisInput to pull data from. |
| Endpoint URL | Enter the endpoint of the Kinesis service to be used. For example, *https://kinesis.us-east-1.amazonaws.com*. More valid Kinesis endpoint URLs<br>can be found at http://docs.aws.amazon.com/general/latest/gr/rande.html#ak\_region. |
| Explicitly set authentication<br>parameters | Select this check box to use the explicit authentication mechanism to connect to Kinesis.<br>Note that this mechanism is supported by Spark V1.4+ only.<br><br>Since this security mechanism requires the AWS Region parameter to be explicitly set, you<br>need to enter the region value to be used in the Region<br>field that is displayed. For example, *us-west-2*.<br><br>It is recommended to use the explicit authentication to gain better security when the<br>Spark version you are using supports this mechanism. With this check box selected, the<br>access credentials are provided directly to Kinesis.<br><br>While if you leave this check box clear, an older authentication mechanism is used. This<br>way, the access credentials are used by Spark as context variables for Kinesis<br>connection. |

## Advanced settings

| Properties | Description |
| --- | --- |
| Checkpoint interval | Enter the time interval (in millisecond) at the end of which tKinesisInput saves the position of its read in the Kinesis stream.<br><br>Data records in a Kinesis stream are grouped into partitions (shards in terms of Kinesis)<br>and indexed with sequence numbers. A sequence number uniquely identifies the position of a<br>record. For further information about the terms used by Amazon in Kinesis, see http://docs.aws.amazon.com/kinesis/latest/dev/key-concepts.html. |
| Initial position stream | Select the starting position to read data from the stream in the absence of the Kinesis<br>checkpoint information.<br><br><ul> <li> <p> <span>Start with the oldest data</span>: starts from the                    beginning of the stream within the limit of 24 hours.</p> </li> <li> <p> <span>Start after the most recent data</span>: starts at                    the position after the latest data of the stream.</p> </li> </ul> |
| Storage level | Select how you want the received data to be cached. For further information about the<br>different levels, see https://spark.apache.org/docs/latest/programming-guide.html#rdd-persistence. |

## Usage

| Usage guidance | Description |
| --- | --- |
| **Usage rule** | This component is used as a start component and requires an output link.<br><br>At runtime, this component keeps listening to the stream and reads new messages once they<br>are buffered in this stream.<br><br>This component, along with the Spark Streaming component Palette it belongs to, appears<br>only when you are creating a Spark Streaming Job.<br><br>Note that in this documentation, unless otherwise explicitly stated, a scenario presents<br>only Standard Jobs, that is to say traditional<br>Talend<br>data<br>integration Jobs. |
| Spark Connection | In the Spark<br>Configuration tab in the Run<br>view, define the connection to a given Spark cluster for the whole Job. In<br>addition, since the Job expects its dependent jar files for execution, you must<br>specify the directory in the file system to which these jar files are<br>transferred so that Spark can access these files:<br><br><ul> <li> <div> <span>Yarn mode</span> (Yarn client or Yarn cluster):<ul> <li> <p>When using Google Dataproc, specify a bucket in the                                            <span>Google Storage staging bucket</span>                                        field in the <span>Spark configuration</span>                                        tab.</p> </li> <li> <p>When using HDInsight, specify the blob to be used for Job                                        deployment in the <span>Windows Azure Storage                                            configuration</span> area in the <span>Spark                                            configuration</span> tab.</p> </li> <li>When using Altus, specify the S3 bucket or the Azure                                    Data Lake Storage for Job deployment in the <span>Spark                                        configuration</span> tab.</li> <li> <p>When using on-premises distributions, use                                        the configuration component corresponding to the file system                                        your cluster is using. Typically, this system is HDFS and so                                        use <a href="https://help.qlik.com/talend/r/en-US/8.0/components/hdfs/thdfsconfiguration">tHDFSConfiguration</a>.</p> </li> </ul> </div> </li> <li> <p> <span>Standalone mode</span>: use the                            configuration component corresponding to the file system your cluster is                            using, such as <a href="https://help.qlik.com/talend/r/en-US/8.0/components/hdfs/thdfsconfiguration-properties-for-apache-spark-batch">tHDFSConfiguration Apache Spark Batch</a> or <a href="https://help.qlik.com/talend/r/en-US/8.0/components/amazon-s3/ts3configuration-properties-for-apache-spark-batch">tS3Configuration Apache Spark Batch</a>.</p> <p>If you are using Databricks without any configuration component present                            in your Job, your business data is written directly in DBFS (Databricks                            Filesystem).</p> </li> </ul><br><br>This connection is effective on a per-Job basis. |
| Limitation | Due to license incompatibility, one or more JARs required to use this component<br>are not provided. You can install the missing JARs for this particular component by<br>clicking the Install button on the Component tab view. You can also find out and add all missing<br>JARs easily on the Modules tab in the Integration perspective of Talend Studio. For<br>details, see [Installing external modules](https://help.qlik.com/talend/r/en-US/8.0/studio-user-guide/installing-external-modules). |