tReplicate properties for Apache Spark Structured Streaming

A schema is a row description. It defines the fields (columns) processed by the component. When you create a Spark Job, avoid the reserved word line when naming fields.

• Built-In: create and store the schema locally for this component only.

• Repository: reuse a schema created and stored in the Repository across projects and Job designs.

Click Edit schema to modify the schema. If you modify a Repository schema, the available options include:

• View schema: view the schema without modifying it.

• Change to built-in property: change the schema to Built-in for local changes.

• Update repository connection: modify the Repository schema and choose whether to propagate the changes to other Jobs.

This component is used as an intermediate step.

This component, along with the Spark Structured Streaming component Palette it belongs to, appears only when you are creating a Spark Structured Streaming Job.

You need to use the Structured Streaming Configuration tab in the Run view to define the connection to a Spark cluster for the whole Job.

This connection is effective on a per-Job basis.