Data Mapping

Some data flow processes are not harvestable using the bridges provided for import. Of course, if these processes are not modeled, it will leave gaps in the lineage and impact analysis answers and provide an incomplete picture of the physical architecture of your systems.

In order to address these gaps and produce proper lineage and impact results, the product has a data mapping editing and management toolset. Data mappings] are essentially simply high-level logical (or notional) definitions of the way data “flows” from some number of source models into elements of a target model. These mappings are specified using a simple web based drag and drop type mapping specification editor and are defined using descriptive text and one may also define pseudo operations using an operation editor.

The data mapping forms a model within the product which defines mappings between source and target data stores. The data stores themselves are modeled separately as models (generally imported object) but external to the data mapping.

A data mapping model may contain many (any number of) mappings. There are two types of mappings which a data mapping might contain:

• Query mapping - is most flexible and you define a column-by-column mapping definition for all the columns in the target table. They may include joins, filters, transformations, etc. Each query mapping is defined for one target classifier.
• Replication mapping – is useful when modeling data replication technologies. Each replication mapping is defined for one target schema and one source schema.

When defining a query mapping, the target classifier may come from any data store model defined in the configuration. Sources too may come from any data store model(s), and as you may have many source classifiers, they could come from multiple source models in the same mapping.

In addition to defining mapping specifications, the data mapping allows you to define all the physical data integration constructs, including joins, filters, transformation operations, etc., which may even be forward engineered into data integration tools.

You may export a data mapping model along with all of its query and replication mappings to a CSV data mapping script format. The format is quite similar to SQL and very simple to understand. You may also edit and re-import those data mappings script format files either back into a data mapping model or as an imported ETL/DI type model, ready to be stitched to the sources and targets.

Finally, the data mapping model can export these mapping specifications to the Excel metadata format. In this way, you may export to this format for reporting purposes.

In addition, one may import the Excel metadata format spreadsheets as imported models which may then be stitched to the sources and targets.