tWriteAvroFields properties for Apache Spark Structured Streaming

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

The schema of this component is read-only. You can click Edit schema to view the schema.

This read-only schema of tWriteAvroFields receives data as an entire object from the schema of its input component without caring about what this input schema should look like and serializes the incoming object into Avro binaries.

That is to say, it does not require the input flow to have the identical schema. For example, an input schema composed of a user column and an age column can be directly serialized. Note that the supported data types by this component are listed in its Basic settings view.

This component is used as an intermediate step.

This component, along with the Spark Structured Streaming component Palette it belongs to, appears only when you are creating a Spark Structured Streaming Job.

Note that in this documentation, unless otherwise explicitly stated, a scenario presents only Standard Jobs, that is to say traditional Talend data integration Jobs.

You need to use the Structured Streaming Configuration tab in the Run view to define the connection to a Spark cluster for the whole Job.

This connection is effective on a per-Job basis.