What's new in R2021-10

You can now use Azure Synapse Analytics with Apache Spark pools as a new distribution for your Spark Batch and Spark Streaming Jobs with Spark 3.0 in YARN cluster mode only. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard. For more information, see Defining the Azure Synapse Analytics connection parameters.

Azure Synapse Analytics allows you to process your data thanks to various analytics engines. With Apache Spark pools you can have various compute capabilities (such as speed and efficiency) and a compatibility with ADLS Gen2 storage.

This feature is now generally available.

All Talend products with Big Data

• tHiveWarehouseConfiguration: enables the reuse of the Hive Warehouse Connector connection configuration to Hive in the same Job.
• tHiveWarehouseInput: extracts data from Hive and sends the data to the component that follows using Hive Warehouse Connector.
• tHiveWarehouseOutput: connects to a given Hive database and writes the received data into a given Hive table or a directory in HDFS using Hive Warehouse Connector.

With Hive Warehouse Connector, Talend Studio support Hive transactional managed tables which allows you to have a more optimal transaction control over your data.

All Talend products with Big Data

All Talend products with Talend Studio

The tSnowflakeOutputBulk, tSnowflakeBulkExec, and tSnowflakeOutputBulkExec components provide the S3 assume role option, which allow you gaining the access permissions associated to an AWS IAM role.

All Talend products with Talend Studio

The tSnowflakeBulkExec and tSnowflakeOutputBulkExec components provide the Snowflake access to storage drop-down list, which is used for COPY command authentication.

All Talend products with Talend Studio

The tFileOutputExcel component provides the Streaming append option, which, when selected, allows you to write data to an existing file in the streaming mode. This option helps improve performance when you write large volume of data with a limited memory size.

All Talend products with Talend Studio

The tSalesforceInput and tSalesforceOutput components provide the Use UTC time for datetime option, which, when selected, uses UTC time for columns that are of datetime type. If this option is not selected, the local time is used.

All Talend products with Talend Studio

This release provides the following new components:

• tFileInputORC and tFileOutputORC, which provide access to files that are of ORC format.
• tCyberarkInput, which retrieves CyberArk-managed login information of the specified application and passes the information to the next component that requires it. By preventing exposure of credential information, this component helps improve the security of managed applications.

All Talend products with Talend Studio

The Include shared folders/files and Include shared drives options have been added to the Advanced settings view and allow you to access shared files, folders and drives. The Corpora type option has been added to narrow the scope of file and folder searches.

These options are available for tGoogleDriveCopy, tGoogleDriveCreate, tGoogleDriveDelete, tGoogleDriveGet, tGoogleDriveList and tGoogleDrivePut.

All Talend products with Talend Studio

The Use query option has been added to the Advanced settings view and allows you to search for specific files and folders.

All Talend products with Talend Studio

All Talend Platform and Data Fabric products