Skip to main content

Schema change detection for JDBC addressed and registered entities

Schema change detection has always been supported for addressed entities sourced from JDBC (relational) connections. However, business metadata such as business name and description was not preserved. As of the November 2021 release, business metadata is preserved.

As of the November 2021 release, schema change detection is supported for registered entities sourced from JDBC (relational) connections. New installations will have this feature automatically enabled. Upgraded installations will require the addition of a new property in file core_env.properties:

# If true, for ADDRESSED and REGISTERED entities loaded from JDBC relational sources, column additions and deletions will be detected and applied. If true, business metadata and tags will be preserved and not overwritten. Set to false for legacy behavior. Default: false (default to false so it is disabled for legacy customers that upgrade).
support.schema.change.detection=true

Set the above property to false to restore legacy behavior.

If this feature is enabled, all entity and field properties will survive reload, except entity properties src.file.glob and header.validation.pattern, which must be updated to reflect column additions and deletions. This applies to both external and internal entities and fields.

All field data will survive reload, except for the following elements (as seen in the UI) which are directly derived from the JDBC column: Data Type, Index, Required (NOT NULL), Primary Key, and Foreign Key.

The Load Log history of all prior loads will be kept. However, the sample data for prior loads is deleted as are distribution database table references to the prior loads (for example, from the discover query window). Note also that when initially defining an entity, if all columns are not selected to become fields, the unselected columns are automatically detected on first load. This feature ensures the full set of columns is always captured as fields.

For more information, see also Adding new fields to a managed entity

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!