Skip to main content Skip to complementary content

Dynamic schema

About this task

Talend Studio allows you to add a dynamic column to the schema of certain components in a Job. The dynamic column may constitute the only column in the schema, or it may be added after known columns, as the last column in the schema.

The dynamic column retrieves the columns which are undefined in the schema. This means that source columns which are unknown when the Job is designed, become known at runtime and are added to the schema. This can make Job design much easier as it allows for simple one to one mapping of many columns. There are many uses for dynamic columns. For instance in data migration tasks, developers can copy columns of data to another location without having to map each column individually.

Note that any static object set in the schema editor, such as data pattern or default value, is not taken into account for a dynamic column.

Information noteWarning: The Dynamic Schema functionality is to help you configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. However, it is not a guarantee that your schema will stick 100% accurately to the schema of the actual data to handle.

While the dynamic schema feature significantly eases Job designs, it does not work in all components. For a list of components that support this feature, go to <install_dir>/plugins/, where <install_dir> is the Talend Studio installation directory, extract the jar file org.talend.core.tis_<version>.jar to get the text file supportDynamic.txt in the resources folder.

Information noteWarning: In the Database Input components, the SELECT query must include the * wildcard, to retrieve all of the columns from the table selected.

For further information about defining dynamic schemas, see Defining dynamic schema columns.

For further information regarding the mapping of dynamic columns, see Mapping dynamic columns.

For an example of using the dynamic schema feature in a Job, see Dynamic Schema.

Defining dynamic schema columns

The dynamic schema columns are easy to define. To define dynamic columns for the Databases Input and Output components, or for tFileInputDelimited and tFileOutputDelimited.

Procedure

  1. In the component's Basic settings tab, set the Property Type as Built-In.
    Information noteWarning: The dynamic schema is only supported in Built-In mode.
  2. Click Edit schema to define the schema.
    The Schema dialog box opens.
    Schema dialog box.
  3. In the last row added to the schema, enter a name for the dynamic column in the Column field.
    Information noteWarning: In the Database Input components, the SELECT query must include the * wildcard, to retrieve all of the columns from the table selected.
  4. Click the Type field and then the arrow which appears to select Dynamic from the list.

Mapping dynamic columns

About this task

It is easy to map dynamic columns in the tMap component, as in the Map Editor, simply dropping the dynamic column from the input schema to the output schema does not change any of the column values:

Map Editor.

However, certain limitations must be respected:

  • The dynamic column must be mapped on a one to one basis and cannot undergo any transformations.
  • The dynamic column cannot be used in filter expressions or in variables.
  • The dynamic column cannot be renamed in output tables and cannot be used as a join condition.
Information noteNote: Dynamic schemas can be mapped to several outputs and can also be mapped from lookup inputs.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here