# tS3Configuration properties for Apache Spark Structured Streaming

[Preview](https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_Hub/Introduction/feature-preview-program-overview.htm)

These properties are used to configure tS3Configuration running in the Spark Structured Streaming Job framework.

The Spark Structured Streaming tS3Configuration component belongs to the Storage family.

The streaming version of this component is available in Talend Real-Time Big Data Platform and in Talend Data Fabric.

## Basic settings

| Properties | Description |
| --- | --- |
| Access Key | Enter the access<br>key ID that uniquely identifies an AWS Account. For further information about how to<br>get your Access Key and Secret Key, see [Getting Your AWS Access<br>Keys](http://docs.aws.amazon.com/ses/latest/DeveloperGuide/get-aws-keys.html). |
| Secret Key | Enter the secret access key,<br>constituting the security credentials in combination with the access Key.<br><br>To enter the secret key, click the [...]<br>button next to the secret key field, and then in the pop-up dialog box enter the password<br>between double quotes and click OK to save the<br>settings. |
| Bucket name | Select this check box to specify an S3 bucket name, then enter the name in<br>the field that appears. |
| Temp Folder | Enter the location of the temp folder in S3.<br>This folder will be automatically created if it has not existed by the time of the<br>execution. |
| Assume Role | Select this check box to make the Job temporarily assume an AWS IAM role<br>and its associated permissions. Then specify the following parameters:<br><br><ul> <li><span>Role ARN</span>: enter the Amazon Resource Name of the                        role to assume.</li> <li><span>Role session name</span>: enter a name to identify the                        assumed-role session.</li> <li><span>Session duration (minutes, 15mins~1hour)</span>: enter the                        duration of the session in minutes.</li> <li><span>External ID</span>: enter the external ID if the trust                        policy of the role requires one.</li> </ul> |

## Advanced settings

| Properties | Description |
| --- | --- |
| Set region | Select this check box and select the region<br>to connect to.<br><br>This setting appears only when Assume Role is enabled. |
| Set endpoint | Select this check box and enter the Amazon S3 endpoint to use in the<br>Endpoint field.<br><br>If you leave this check box clear, the default endpoint is used. This setting<br>appears only when Assume Role is enabled. |
| Set region and Set endpoint | If the AWS<br>administrator has enabled the STS endpoints for the regions you want to use for<br>better response performance, select the Set<br>region check box and then select the regional endpoint to be<br>used.<br><br>If the endpoint you<br>want to use is not available in this regional endpoint list, clear the<br>Set<br>region check box, then select the Set<br>endpoint check box and enter the endpoint to be used.<br><br>This service allows you to request<br>temporary, limited-privilege credentials for the AWS user authentication.<br>Therefore, you still need to provide the access key and secret key to<br>authenticate the AWS account to be used.<br><br>For a list of the STS endpoints you can<br>use, see [AWS Security Token Service](https://docs.aws.amazon.com/general/latest/gr/rande.html#sts_region). For<br>further information about the STS temporary credentials, see [Temporary Security Credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html).<br>Both articles are from the AWS documentation. |

## Usage

| Usage guidance | Description |
| --- | --- |
| Usage rule | This component is used with no need to be connected to other components.<br><br>Multiple tS3Configuration components are allowed per Job.<br>The Assume Role option can be added to multiple<br>tS3Configuration components.<br><br>You need to drop tS3Configuration along with the file system related<br>subJob to be run in the same Job so that the configuration is used by the whole Job<br>at runtime.<br><br>This component, along with the Spark Structured Streaming component Palette it belongs to,<br>appears only when you are creating a Spark Structured Streaming Job. |
| Spark Connection | You need to use the Structured Streaming Configuration<br>tab in the Run view to define the connection to a Spark cluster for<br>the whole Job.<br><br>This connection is effective on a per-Job basis. |