Displaying record counts for external entities
Record count is calculated for external entities (representation of external source in source module) from load logs. Due to varying Total Counts resulting from whether Entity Type is Snapshot or Incremental, users are able to select Total Record Counts across all loads and Latest Record Counts for only the most recent load.
Record count options for totals across all loads are: Total Record Count, Good Record Count, Bad Record Count, and Ugly Record Count.
Record count options for the latest load are: Latest Total Record Count, Latest Good Record Count, Latest Bad Record Count, and Latest Ugly Record Count.
These external entity counts are derived from load log record counts across all loads. They are calculated when data load type is DATA, STATISTICS, STALE and the job status is FINISHED.
Note that record counts for data load type Append and Overwrite are not factored into the record counts because in these cases, the original data load log record count is resolved with the Append (added) or Overwrite (replaced) records. Therefore, these counts are already represented in the total counts.
The purpose of offering Latest Record Counts versus Total Record Counts is to offer users a way to differentiate load log totals relative to the Latest Load log, these totals will oftentimes be different. For example, if an entity is loaded twice with 10 records, the Latest Total Record Count will be 10 and the Total Record Count will be 10. If the entity is loaded again with 10 records, Latest Total Record Count will be 10 and Total Record Count (summing both Load Logs) will be 20.
The metrics populate in source entity grids if they have been selected as visible columns in User Preferences. To configure External Entity grid to display record counts:
- Select Profile from username dropdown
- Select User Preferences tab
- Select External Entity for Data Grid value
- Check record count columns to display
Specifying record count columns for discover entity grid
Selected and visible record count columns in source entity grid
Note that internal entity counts (in discover module) will most always differ from those displaying in external entity (source) grids. This is because the counts for internal entities are calculated from metrics created at the time of data load— the counts displaying on internal entities are the net counts that users will see when they query hive (in contrast, external entity counts are derived from load logs). For more information on how record counts for internal entities are derived, see Displaying record counts for internal entities.