Supported Cloud Big Data platform distribution versions for Talend Jobs

Cloud Hadoop distributions

Talend supports the following cloud platforms for Big Data. Click your cloud platform to see the Big data support information.

Old versions of the supported Big Data platforms are being retired by their vendors. Talend ceases to support a version once this version reaches its date of end of support set by its vendor.

Talend and its community provide you with the convenience to keep using a version its vendor ceases to support in Talend products. For this reason, this version can be still listed in the following tables and available in the products but Talend stops providing support for this version.

Talend supports the minor versions of the platform versions listed in the following tables.

Amazon EMR
Amazon EMR version	Supported frameworks	Supported Hadoop elements in Spark batch	Supported Hadoop elements in Spark streaming	Supported Hadoop elements in Standard
v4.5.0 (Hadoop 2.7.2)	Standard Spark v1.6 (Deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v4.6.0 (Hadoop 2.7.2)	Standard Spark v1.6 (Deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v5.0.0 (Hadoop 2.7.2)	Standard Spark v2.0 (deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v5.5.0 (Hadoop 2.7.3)	Standard Spark v2.1 (deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v5.8.0 (Hadoop 2.7.3)	Standard Spark v2.2 (deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v5.15.0 (Hadoop 2.8.3)	Standard Spark v2.3 (deprecated)	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v5.29.0 (Hadoop 2.8.5)	Standard Spark v2.4	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v6.2.0 (Hadoop 3.2.1)	Standard Spark v3.0	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop

The supported Amazon EMR versions for the tAmazonEMRManage component are 4.5.0, 4.6.0, 4.9.2, 5.11.0, 5.15.0 and 5.29.0.

Cloudera Altus on AWS for Big Data
Cloudera version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
CDH5.11 (deprecated)	Spark v2.1 (deprecated)	S3 Redshift DynamoDB	S3 Kinesis Redshift DynamoDB

Cloudera Altus on Azure for Big Data
Cloudera version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
CDH5.11 (deprecated)	Spark v2.1 (deprecated)	ADLS Gen1 Azure Blob Storage HDFS	ADLS Gen1 Azure Blob Storage HDFS

Databricks on AWS for Big Data
Databricks on AWS version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
3.5 LTS (deprecated)	Standard Spark v2.2 (deprecated)	Hive S3 DynamoDB	Hive S3 DynamoDB Kinesis	DBFS
5.5 LTS	Standard Spark v2.4	Hive S3 DynamoDB Snowflake MongoDB TDM components as beta tDataprepRun	Hive S3 DynamoDB Kinesis Snowflake MongoDB TDM components as beta tDataprepRun	DBFS
6.4	Standard Spark v2.4.5	Azure Blob Storage ADLS Gen2 Snowflake DeltaLake	Azure Blob Storage ADLS Gen2	DBFS
7.3 LTS	Standard Spark v3.0.1	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	DBFS

Databricks on Azure for Big Data
Databricks on Azure version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
3.5 LTS (deprecated)	Standard Spark v2.2 (deprecated)	Hive Azure Blob Storage ADLS Gen1	Hive Azure Blob Storage ADLS Gen1	DBFS
5.5 LTS	Standard Spark v2.4	Hive Azure Blob Storage ADLS Gen1 ADLS Gen2 Snowflake DeltaLake MongoDB TDM components as beta tDataprepRun	Hive Azure Blob Storage ADLS Gen1 ADLS Gen2 Snowflake DeltaLake MongoDB TDM components as beta tDataprepRun	DBFS
6.4	Standard Spark v2.4.5	Azure Blob Storage ADLS Gen2 Snowflake DeltaLake	Azure Blob Storage ADLS Gen2	DBFS
7.3 LTS	Standard Spark v3.0.1	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	DBFS

Google Dataproc for Big Data
Google Dataproc version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
v1.1 (deprecated)	Standard Spark v2.0 (deprecated)	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON Google PubSub	Hive
v1.4 (deprecated)	Standard Spark v2.4	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON Google PubSub	Hive

Microsoft HD Insight for Big Data
Microsoft HD Insight version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
3.4 (deprecated)	Spark v1.6 (deprecated)	Hive	Hive	-
3.6 (deprecated)	Spark v2.1 (deprecated)	Hive	Hive	-
4.0	Spark v2.3 (deprecated) and v2.4	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive

Qubole on AWS for Big Data
Qubole version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
Qubole Spark 2 (deprecated)	Standard Spark v2.2 (deprecated)	Redshift S3 DynamoDB	Redshift S3 DynamoDB Kinesis	S3 Hive

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here