What's new in R2020-08
Big Data: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Lightweight dependencies for CDH 6.x |
When you run a Job on CDH 6.x distributions, you can
reduce the time spent by the Job to be launched when you select the
Use lightweight dependencies
check box from the Spark
configuration tab in the Run view. It reduces the number of libraries to only
the Talend libraries. This could prevent issues about dependencies,
missing signature, wrong JAR version or missing JAR for example. With
this option, you can use another classpath, different from the Cloudera
default one, by selecting the Use custom
classpath check box and entering the JARs you want to
use in a regex syntax separated by a comma.
|
All Talend products with Big Data |
| Customizing precision in the schema for output components | You can now select a precision different from the
standard for the BigDecimal type when you update the output schema for the
following components:
|
All Talend products with Big Data |
| tS3Configuration: setting the name of the DynamoDB table in EMRFS | When you use the EMRFS consistent view option, you can enter the name of the metadata DynamoDB table to be used. |
All Talend products with Big Data |
| tDeltaLakeInput and tDeltaLakeOutput: new path available to store the data | You can specify an external path to another filesystem
different from the DBFS (ADLS Gen2 or S3) to store the data.
|
All Talend products with Big Data |
| tDeltaLakeOutput: new operations provided in the Action property | You can drop a table which results in the deletion and
recreation of the table. You can also truncate a table which results in the
deletion of data while the schema remains.
|
All Talend products with Big Data |
| tDeltaLakeOutput: new optimize property provided in the Basic settings view | You can optimize the layout of Delta Lake data on
Databricks.
|
All Talend products with Big Data |
| Using a Hadoop configuration file with Spark Batch and Spark Streaming Jobs | You can connect Spark Batch and Spark Streaming Jobs to a Hadoop cluster in the Repository using a configuration JAR file. You specify the path to this file either in the Spark Configuration of the Job or in the Hadoop cluster configuration. This option is available for both Yarn cluster and Yarn client on non-Cloud distributions. Optionally, you can contextualize this connection parameter to automatically connect to the right cluster based on the environment in which you run the Job. |
All Talend products with Big Data |
|
Support for High Availability for EMR 5.23 or later |
High availability is available when you run Talend Jobs with the Amazon EMR distribution in version 5.23 or later. You can now have multiple master nodes in your cluster. |
All Talend products with Big Data |
Data Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
| tDataprepRun enhancement | The tDataprepRun component now shows an error message
when creating a new preparation with dynamic schema.
|
All Talend products except Talend ESB |
| tELTMap enhancement | In the ELT Map Editor of the tELTMap component, you can
now enter a multi-line expression for an output column and access proposals
including input columns, output columns, and context variables by pressing
Ctrl+Space in a new pop-up dialog box.
|
All Talend products with Talend Studio |
| POM generation enhancement | A new option Exclude deleted
items is available for generating the POM file for Maven
build. With this option selected, the modules for the deleted items will be
excluded from the POM file of current project, and the source for the
deleted test cases will not be generated. Information noteNote: You need to resynchronize the POM file to apply the new
settings of this option.
|
All Talend products with Talend Studio |
| Talend type mapping enhancement | You can now set the default pattern for each date type
in Talend type mapping
file. This allows the date pattern for the date type columns to be set
automatically when retrieving or guessing schema from a table.
|
All Talend products with Talend Studio |
|
tSSH library upgraded |
Ganymed is now deprecated and the component now supports a new library: Apache mina-sshd. |
All Talend products with Talend Studio |
|
tSalesforceInput: new query mode provided |
The tSalesforceInput component provides the BulkV2 query mode, which allows you to query even larger amount of data. The component also provides the Split query results into small sets option for the BulkV2 mode, allowing you to split query results into sets that are of specific size. 
|
All Talend products with Talend Studio |
|
Formatting date using date pattern defined in schema |
The tSnowflakeOutput and the tSnowflakeOutputBulkExec components provide the Use schema date pattern option, allowing you to format dates using the date pattern defined in the schema. 
|
All Talend products with Talend Studio |
|
tSnowflakeInput: option renamed and improved |
The Allow snowflake to convert columns and tables to uppercase option changes to Use unquoted object identifiers, with the function improved. 
|
All Talend products with Talend Studio |
|
tFTPRename: table column name fixed |
The tFTPRename component supports only filenames in the Files field and the column name Filemask changes to Filename. 
|
All Talend products with Talend Studio |
|
tS3Connection: path-style access supported |
The tS3Connection component provides support for path-style access. 
|
All Talend products with Talend Studio |
|
tMongoDBOutput: action on data fields customizable |
For Upsert with set action, you can specify whether or not a field can be updated/inserted. 
|
All Talend products with Talend Studio |
Data Quality: new features
|
Feature |
Description |
Available in |
|---|---|---|
| Phone number standardization | Phone numbers can now be validated for a given region:
The Google libphonenumber library has also been updated to the most recent version. |
All Talend Platform and Data Fabric products |
Application Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
|
Microservices |
Camel metrics are now exposed to Prometheus in
Microservices to monitor the execution of Routes, JVM memory, CPU
consumption, and so on.
|
All Talend products with ESB |
Continuous Integration: new features
|
Feature |
Description |
Available in |
|---|---|---|
| POM file generation - new parameter | The mvn org.talend.ci:builder-maven-plugin:7.3.3:generateAllPoms command allows you to re-generate all .pom files of a project before building it. It is also useful in case you want to test a new version of the product before migrating it. |
All Talend products with Talend Studio |
| Custom script - new parameter | The mvn org.talend.ci:builder-maven-plugin:7.3.3:executeScript command allows you to write your own script with CommandLine commands and execute it at build time. |
All Talend products with Talend Studio |
| Camel metrics exposure to Prometheus - new parameter | You are now able to use the -Dstudio.prometheus.metrics=true parameter while publishing ESB artifacts to Docker in order to expose Camel metrics to Prometheus and get more details about the deployed Routes. |
All Talend products with ESB |
| Debug mode - new parameter | You are now able to use the -Dstudio.talendDebug=true parameter to get additional logs. This parameter could be useful when trying to debug build issues with the support team. |
All Talend products with Talend Studio |
| Build improvement | Depending on your Talend Studio project settings, all project items that are located in the recycle bin can now be excluded from the Continuous Integration build. |
All Talend products with Talend Studio |