tS3List

Either Built-In or Repository.

Built-In: No property data stored centrally.

Repository: Select the repository file where the properties are stored.

Select this check box and in the Component List drop-down list, select the desired connection component to reuse the connection details you already defined.

Credential provider

Specify the way to obtain AWS security credentials.

• Static credentials: Use access key and secret key as the AWS security credentials.
• Inherit credentials from AWS role: Obtain AWS security credentials from your EMR instance metadata. To use this option, the Amazon EMR cluster must be started and your Job must be running on this cluster. For more information, see Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances. This option enables you to develop your Job without having to put any AWS keys in the Job, thus easily comply with the security policy of your organization.
• Web Identity Token: Use Web token for establishing the Amazon S3 connection.

• Profile credentials or inherit from AWS role: Use credentials stored in a file (known as profile configuration file) as the AWS security credentials or inherit AWS security credentials from the AWS role.

  AWS security credentials can be grouped in profiles in profile configuration files. A profile configuration file can contain one or multiple profiles. You can optionally specify the profile configuration file in the Profile path field and specify the profile to be used in the Profile name field.

  If the Profile path field and the Profile name filed are left empty, <user folder>/.aws/credentials will be used as the profile configuration file and the profile named default will be used.

  In cases of EC2 instances, if the profile (the specified or the default profile) does not exist, the component will try to inherit the AWS security credentials from the AWS role.

This option can work as a dynamic parameter and be assigned a value using a context variable.

Assume role

If you temporarily need some access permissions associated to an AWS IAM role that is not granted to your user account, select this check box to assume that role. Then specify the values for the following parameters to create a new assumed role session.

Ensure that access to this role has been granted to your user account by the trust policy associated to this role. If you are not certain about this, ask the owner of this role or your AWS administrator.

Specify the AWS region by selecting or entering a region name between double quotation marks (e.g. "us-east-1"). For more information about the AWS Region, see Regions and Endpoints.

Key prefix: Enter the prefix of files to be listed. This way, only files with that prefix will be listed.

Bucket name: Enter the name of the bucket, namely the top level folder, whose files will be listed.

Key prefix: Enter the prefix of files to be listed.

Not available when List all bucket objects is selected.

This option can work as a dynamic parameter and be assigned a value using a context variable.

Select the AWS region of the STS service. If the region is not in the list, you can enter its name between double quotation marks. The default value is us-east-1.

This drop-down list is available only when the Assume role check box is selected.

Specify STS endpoint

Select this check box to specify the AWS Security Token Service (STS) endpoint from which to retrieve the session credentials. For example, enter sts.amazonaws.com.

This service allows you to request temporary, limited-privilege credentials for the AWS user authentication. Therefore, you still need to provide the access key and secret key to authenticate the AWS account to be used.

For a list of the STS endpoints you can use, see AWS Security Token Service. For further information about the STS temporary credentials, see Temporary Security Credentials. Both articles are from the AWS documentation.

This check box is available only when the Assume role check box is selected.

If the administrator of the account to which the role belongs provided you with an external ID, enter its value here. The External Id is a unique identifier that allows a limited set of users to assume the role.

This field is available only when the Assume role check box is selected.

When you assume a role, the trust policy of this role might require Multi-Factor Authentication (MFA). In this case, you must indicate the identification number of the hardware or virtual MFA device that is associated with the user who assumes the role.

This field is available only when the Assume role check box is selected.

When you assume a role, the trust policy of this role might require Multi-Factor Authentication (MFA). In this case, you must indicate a token code. This token code is a time-based one-time password produced by the MFA device.

This field is available only when the Assume role check box is selected.

List session tags in the form of key-value pairs. You can then use these session tags in policies to allow or deny access to requests.

Transitive: select this check box to indicate that a tag will persist to the next role in a role chain.

For more information about tags, see Passing Session Tags in AWS STS

This field is available only when the Assume role check box is selected.

Enter the Amazon Resource Names (ARNs) of the IAM managed policies that you want to use as managed session policies. Use managed session policies to limit the permissions of the session. The policies must exist in the same account as the role. The resulting session's permissions are the intersection of the role's identity-based policy and the session policies.

For more information about session policies, see the corresponding section in Policies and Permissions

This field is available only when the Assume role check box is selected.

Enter an IAM policy in JSON format that you want to use as a session policy. Use session policies to limit the permissions of the session. The resulting session's permissions are the intersection of the role's identity-based policy and the session policies.

For more information about session policies, see the corresponding section in Policies and Permissions

This field is available only when the Assume role check box is selected.

Config client

• Client parameter: Click the cell and select the client parameter from the list.

• Value: Enter the value for the selected parameter.

This check box is available only when the Use an existing connection check box is cleared.

For related information, read the AWS documentation.

CURRENT_BUCKET_NAME

The current bucket name. This is a Flow variable and it returns a string.

CURRENT_KEY

The current key. This is a Flow variable and it returns a string.

NB_BUCKET

The number of buckets. This is an After variable and it returns an integer.

NB_BUCKET_OBJECT

The number of objects in all buckets. This is an After variable and it returns an integer.

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is selected.

CURRENT_SIZE

The size of the current file (in bytes). This is a Flow variable and it returns an integer number.

CURRENT_LASTMODIFIED

The date the current file was last modified. This is a Flow variable and it returns a date (in the format of MM-DD-YYYY).

CURRENT_OWNER

The owner name of the current object. This is a Flow variable and it returns a string.

CURRENT_OWNER_ID

The owner's AWS ID of the current object. This is a Flow variable and it returns a string.

CURRENT_ETAG

The hexadecimal MD5 Hash value (128-bit) of the content of the current object. This is a Flow variable and it returns a string.

CURRENT_STORAGECLASS

The storage class used for the current object. This is a Flow variable and it returns a string.

Click the [+] button to add a row in the table and fill the Code field with a context variable to choose your database connection dynamically from multiple connections planned in your Job. This feature is useful when you need to access database tables having the same data structure but in different databases, especially when you are working in an environment where you cannot change your Job settings, for example, when your Job has to be deployed and executed independent of Talend Studio.

Once a dynamic parameter is defined, the Component List box in the Basic settings view becomes unusable.

For examples on using dynamic parameters, see Reading data from databases through context-based dynamic connections and Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic settings and context variables, see Dynamic schema and Creating a context group and define context variables in it.