Supported Cloud Big Data platform distribution versions for Talend Jobs

Cloud Hadoop distributions

Talend supports the following cloud platforms for Big Data. Click your cloud platform to see the Big data support information.

Old versions of the supported Big Data platforms are being retired by their vendors. Talend ceases to support a version once this version reaches its date of end of support set by its vendor.

Talend and its community provide you with the convenience to keep using a version its vendor ceases to support in Talend products. For this reason, this version can be still listed in the following tables and available in the products but Talend stops providing support for this version.

Talend supports the minor versions of the platform versions listed in the following tables.

Amazon EMR
Amazon EMR version	Supported frameworks	Supported Hadoop elements in Spark batch	Supported Hadoop elements in Spark streaming	Supported Hadoop elements in Standard
v5.29.0 (Hadoop 2.8.5)	Standard Spark 2.4	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v6.2.0 (Hadoop 3.2.1)	Standard Spark 3.0	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive Sqoop
v6.3.x (Hadoop 3.2.3)	Spark Universal 3.1.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.4.x (Hadoop 3.2.3)	Spark Universal 3.1.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.5.x (Hadoop 3.2.3)	Spark Universal 3.1.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.6.0 (Hadoop 3.2.3)	Spark Universal 3.2.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.7.0 (Hadoop 3.2.3)	Spark Universal 3.2.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.8.0 (Hadoop 3.3.1)	Spark Universal 3.3.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.9.0 (Hadoop 3.3.1)	Spark Universal 3.3.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive
v6.10.0 (Hadoop 3.3.1)	Spark Universal 3.3.x	HBase HDFS HCatalog Hive	HBase HDFS HCatalog Hive	HBase HDFS Hive

The supported Amazon EMR versions for the tAmazonEMRManage component is 5.29.0.

Databricks on AWS for Big Data
Databricks on AWS version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
Availability-noteDeprecated 5.5 LTS	Standard Spark 2.4	Hive S3 DynamoDB Snowflake MongoDB TDM components tDataprepRun	Hive S3 DynamoDB Kinesis Snowflake MongoDB TDM components as beta tDataprepRun	DBFS
Availability-noteDeprecated 6.4	Standard Spark 2.4.5	Azure Blob Storage ADLS Gen2 Snowflake DeltaLake	Azure Blob Storage ADLS Gen2	DBFS
Availability-noteDeprecated 7.3 LTS	Standard Spark 3.0.1	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	DBFS
8.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
9.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
10.x	Spark Universal 3.2.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
11.x	Spark Universal 3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
12.x	Spark Universal 3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
13.x	Spark Universal 3.4.x	Avro Delimited DQ components Hive JSON Parquet Positional S3 XML	Avro Delimited DQ components Hive JSON Parquet Positional S3 XML	None

Databricks on Azure for Big Data
Databricks on Azure version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
Availability-noteDeprecated 5.5 LTS	Standard Spark 2.4	Hive Azure Blob Storage ADLS Gen1 ADLS Gen2 Snowflake DeltaLake MongoDB TDM components tDataprepRun	Hive Azure Blob Storage ADLS Gen1 ADLS Gen2 Snowflake DeltaLake MongoDB TDM components tDataprepRun	DBFS
Availability-noteDeprecated 6.4	Standard Spark 2.4.5	Azure Blob Storage ADLS Gen2 Snowflake DeltaLake	Azure Blob Storage ADLS Gen2	DBFS
Availability-noteDeprecated 7.3 LTS	Standard Spark 3.0.1	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	DBFS
8.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
9.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
10.x	Spark Universal 3.2.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
11.x	Spark Universal 3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
12.x	Spark Universal 3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
13.x	Spark Universal 3.4.x	ADLS Gen2 Avro Delimited DQ components Hive JSON Parquet Positional XML	Avro Delimited DQ components Hive JSON Parquet Positional XML	None

Databricks on Google Cloud Platform (GCP) for Big Data
Databricks on GCP version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
8.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
9.x	Spark Universal 3.1.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
10.x	Spark Universal 3.2.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
11.x	Spark Universal v3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen1 ADLS Gen2 Azure Blob Storage	None
12.x	Spark Universal 3.3.x	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Azure Blob Storage	Hive S3 DynamoDB Snowflake MongoDB DeltaLake ADLS Gen2 Kinesis Azure Blob Storage	None
13.x	Spark Universal 3.4.x	ADLS Gen2 Avro Delimited DQ components Hive JSON Parquet Positional XML	Avro Delimited DQ components Hive JSON Parquet Positional XML	None

Google Dataproc
Google Dataproc version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
2.0.x	Standard Spark Universal 3.1.x	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON Google PubSub	Hive
2.1	Spark Universal 3.3.x	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON	Hive BigQuery Google Storage Avro Delimited Parquet Positional XML JSON Google PubSub	Hive

Microsoft HDInsight for Big Data
Microsoft HDInsight version	Supported frameworks	Supported elements in Spark batch	Supported elements in Spark streaming	Supported elements in Standard
4.0	Spark 2.3 (deprecated) and 2.4	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive
5.0	Spark Universal 3.1.x	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive
5.1	Spark Universal 3.3.x	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive	ADLS Gen2 Azure Blob Storage Hive

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here