Record format matching
Data type
When a user applies a transform control to a source entity the record format of that entity output port advances to become the record format of the input port of the target entity. For the most part, the field data types on either side of the transform will match though some functions require that the data type changes as a function of the operator.
|
Cast Functions |
|
|
Boolean CASE Operators |
Where if a boolean condition is met, these values convert to true/false and stage to the next function. |
|
Boolean Functions |
Data Type STRING, DOUBLE, LONG convert to BOOLEAN type. Examples are some date functions and Boolean functions that include the string 'IS': DateIsBetweenExclusive, DateIsBetweenInclusive, DateIsEqual, DateIsValidFormat, IS_DOUBLE, IS_INTEGER, IS_LONG, IS_NULL, IS_NULL_OR_EMPTY |
Example of a boolean expression
|
Shared common fields must be of the same data type for a Join to validate. It is sometimes possible to use a casting function to transform a field data type (ex. DOUBLE to STRING) for this scenario so that the data types match. |
|
|
All fields must be the same data type on either side of the control. |
Field names
It is a good practice to rename field names to indicate how the value has been transformed. In the case of some transform controllers such as Aggregator where the same field can being queried multiple times, fields must be renamed.
|
Shared common fields can have different names. |
|
|
Unions require identical field names in the source entities being united. |
Transform example: Field names can be renamed within the controller
Many errors in prepare around field names can be avoided or resolved by renaming target fields that have duplicate names and then disconnecting and reconnecting the target entity if it has not executed; this step refreshes the target entity if it has executed (and data had been loaded to the entity).
Mismatched field errors are oftentimes the result of new data being loaded to a target entity with record format in place from previous transform definition. Once reconnected, updated field names in target entity display.
Field counts will not always be the same on both sides of the transform control. For example, a transform operation may delete fields resulting in the target entity having fewer fields than the source entity. Some errors can be re-mediated by deleting the connector and reconnecting the control to the target entity.
|
Source entities can have different field counts. |
|
|
Field counts must match for both source entities. |
