tS3Copy Standard properties

Use an existing connection

Select this check box and in the Component List drop-down list, select the desired connection component to reuse the connection details you already defined.

Access Key

Specify the Access Key ID that uniquely identifies an AWS Account. For how to get your Access Key and Access Secret, visit Getting Your AWS Access Keys.

Secret Key

Specify the Secret Access Key, constituting the security credentials in combination with the access Key.

To enter the secret key, click the [...] button next to the secret key field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

Credential Provider

Specify the way to obtain AWS security credentials.

• Static Credentials: Use access key and secret key as the AWS security credentials.
• Inherit credentials from AWS role: Obtain AWS security credentials from your EMR instance metadata. To use this option, the Amazon EMR cluster must be started and your Job must be running on this cluster. For more information, see Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances. This option enables you to develop your Job without having to put any AWS keys in the Job, thus easily comply with the security policy of your organization.
• Web Identity Token: Use Web token for establishing the Amazon S3 connection.

• Profile credentials or inherit from AWS role: Use credentials stored in a file (known as profile configuration file) as the AWS security credentials or inherit AWS security credentials from the AWS role.

  AWS security credentials can be grouped in profiles in profile configuration files. A profile configuration file can contain one or multiple profiles. You can optionally specify the profile configuration file in the Profile config file path field and specify the profile to be used in the Profile name field.

  If the Profile config file path field and the Profile name filed are left empty, <user folder>/.aws/credentials will be used as the profile configuration file and the profile named default will be used.

  In cases of EC2 instances, if the profile (the specified or the default profile) does not exist, the component will try to inherit the AWS security credentials from the AWS role.

This option can work as a dynamic parameter and be assigned a value using a context variable.

Assume Role

If you temporarily need some access permissions associated to an AWS IAM role that is not granted to your user account, select this check box to assume that role. Then specify the values for the following parameters to create a new assumed role session.

Region

Specify the AWS region by selecting a region name from the list or entering a region between double quotation marks (e.g. "us-east-1") in the list. For more information about the AWS Region, see Regions and Endpoints.

Source Bucket

Specify the name of the source bucket that contains the object to be copied.

Source Key

Specify the path to the object to be copied.

Destination Bucket

Specify the name of the destination bucket to which the object will be copied.

Destination Key

Specify the path to the destination object, using slash (/) to separate folders in the path.

If:

• you only specify a file name in this field, the file will be in the bucket;

• the path contains folders that do not exist, the folders will be created;

• the file already exists, it will be overwritten.

Select this check box to enable server-side encryption with Amazon S3-Managed Encryption Keys (SSE-S3) and use the 256-bit Advanced Encryption Standard (AES-256) cipher to encrypt your data.

For more information about the server-side encryption, see Protecting Data Using Server-Side Encryption.

Select this check box to enable server-side encryption with AWS KMS-Managed Keys (SSE-KMS) instead of Amazon S3-Managed Encryption Keys (SSE-S3).

This property is available when the Server-Side Encryption check box is selected.

Specify your own customer master key (CMK) that is created in the IAM console using AWS Key Management Service for SSE-KMS encryption. If not specified, the default CMK, which is created the first time you add an SSE-KMS encrypted object to the defined bucket in your region, will be used for SSE-KMS encryption.

This property is available when the Use KMS check box is selected.

Die on error

Select this check box to stop the execution of the Job when an error occurs.

Clear the check box to skip any rows on error and complete the process for error-free rows.

Config client

Select this check box and specify the client paramter(s) by clicking the [+] button to add as many rows as needed, each row for a client parameter, and then setting the value of the following fields for each parameter:

• Client Parameter: click the cell and from the drop-down list displayed select the client parameter.

• Value: enter the value for the selected parameter.

This check box is available only when the Use an existing connection check box is cleared.

STS Endpoint

Select this check box to specify the AWS Security Token Service (STS) endpoint from which to retrieve the session credentials. For example, enter sts.amazonaws.com.

This check box is available only when the Assume Role check box is selected.

Select the AWS region of the STS service. If the region is not in the list, you can enter its name between double quotation marks. The default value is us-east-1.

This drop-down list is available only when the Assume Role check box is selected.

If the administrator of the account to which the role belongs provided you with an external ID, enter its value here. The External Id is a unique identifier that allows a limited set of users to assume the role.

This field is available only when the Assume Role check box is selected.

When you assume a role, the trust policy of this role might require Multi-Factor Authentication (MFA). In this case, you must indicate the identification number of the hardware or virtual MFA device that is associated with the user who assumes the role.

This field is available only when the Assume Role check box is selected.

When you assume a role, the trust policy of this role might require Multi-Factor Authentication (MFA). In this case, you must indicate a token code. This token code is a time-based one-time password produced by the MFA device.

This field is available only when the Assume Role check box is selected.

List session tags in the form of key-value pairs. You can then use these session tags in policies to allow or deny access to requests.

Transitive: select this check box to indicate that a tag will persist to the next role in a role chain.

For more information about tags, see Passing Session Tags in AWS STS

This field is available only when the Assume Role check box is selected.

Enter the Amazon Resource Names (ARNs) of the IAM managed policies that you want to use as managed session policies. Use managed session policies to limit the permissions of the session. The policies must exist in the same account as the role. The resulting session's permissions are the intersection of the role's identity-based policy and the session policies.

For more information about session policies, see the corresponding section in Policies and Permissions

This field is available only when the Assume Role check box is selected.

Enter an IAM policy in JSON format that you want to use as a session policy. Use session policies to limit the permissions of the session. The resulting session's permissions are the intersection of the role's identity-based policy and the session policies.

For more information about session policies, see the corresponding section in Policies and Permissions

This field is available only when the Assume Role check box is selected.

Multipart threshold (MB)

Specify the multipart threshold (in MB). An object will be copied to the destination bucket in parts if its size is larger than the multipart threshold.

Part size (Mb) for file larger than 5GB

Specify the part size in MB. This parameter takes effect only when you copy an Amazon S3 object that is larger than 5 GB through multiple parts.

Note that in the copy operation, the object's metadata may drop during transition.

Canned Access Control

Select an option from the drop-down list to grant a predefined permission for the current resource. See Canned ACL for information about canned ACL.

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is selected.

Usage rule

This component can be used as a standalone component.