Using secured Kafka in Talend Studio
You can use secured Kafka with Talend Studio, but not all security options are supported with all versions of Kafka. The matrix of supported versions depends on the kind of job, the Talend version and the Kafka version.
This article provides the matrix of which Kerberos/SSL options are supported with different Kafka and Talend versions.
Environment
- This matrix applies to Talend Studio v6.2 and later.
- The Kafka components on Spark are available only with Talend Real-Time Big Data Platform and Talend Data Fabric subscriptions.
Support Matrix
Supported security options are in the cells.
|
Talend v6.2 |
Talend v6.3 |
||
|---|---|---|---|
|
Kafka 0.9 |
Kafka 0.9 |
Kafka 0.10 | |
|
Standard Jobs |
Kerberos |
Kerberos |
Kerberos |
|
SSL |
SSL |
SSL |
|
|
Spark 1.6 |
- |
- |
- |
| Spark 2.0 |
- |
SSL |
SSL |
Currently, Spark does not have the API required to work with Kerberized Kafka.
To confirm which Kafka version your Spark Job is using, look in the Kafka-related package of the Spark module of your Hadoop distribution. Your Spark Job will use the Kafka version defined there.