Data Flow Overview for Models
Data Integration and ETL/ETL data processes contain lineage within the model, even without stitching them to other models. In addition, data store models such as databases with views and/or stored procedures also present lineage in this fashion.
You may even use the lineage overview on models not in the current configuration using the MANAGE > Repository function.
The Data Flow Overview lineage (see TYPE in the upper right of diagram) has a very limited scope, only to the specific subset that is a self-contained model, e.g., a schema model in a database model or a transformation model or connection model in an ETL/DI model or BI model. Thus, it has none of the information determined in an Impact or Lineage trace diagram, so that connections are not resolved (may just be a * because of “Select *” in the connection definition). To get a true lineage picture you must use the Impact or Lineage trace from a table or column (field).
In addition, in the Overview lineage of a connection model in an ETL/DI model or BI model, many of the object are not even included in an end-to-end lineage trace and thus you will not be able to trace from that object at all. This limitation is caused by the fact that connection definitions themselves only show in the overview and are not a part of a lineage trace and thus there is no way to trace lineage from them.
The data flow overview lineage presents detailed transformation lineage vs. a lineage trace which presents summary lineage.
In particular, when you select a runtime job, and go to the data flow tab, you see the detailed transformation lineage: every transformation is being depicted on the screen. This view is good as long as you only look at one job at a time.
Finally, the Data Flow presentation for overview lineage does notoffer the Tree tab as the scope is only the currently model and the Tree tab features end-to-end lineage, which is not available for overview lineage presentations.
Steps
- Sign in as a user which has at least the Metadata Viewing or Data Managementcapability object role assignment to the model to be analyzed for Overview lineage.
- Open the object page of the model (e.g., ETL/DI or BI model)
- Go to the Data Flow tab.
Example
Sign in as Administrator and go to the object page for the Staging to Dimensional Talend DI model and go to the Data Flow tab.
Right-click on the top DI process name ShippingPOC and select Open.