Assessing data quality

After opening a dataset, you can take a look at several parts of the overview to learn more about its overall quality, its schema, the quality statistics, and semantic types of each columns.

Quality indicators of the dataset

When you open the overview of a dataset that has just been registered, most of the information is grayed out. To calculate the data quality for the first time, click the Compute button. If the quality has already been computed once before, but you want to make sure that the data is up to date, click the Refresh button.

Each compute or refresh in pushdown will induce some costs in your Cloud data warehouse (Snowflake or Databricks). For more information, see Data quality for connection-based datasets.

There are two main sections where the quality is displayed.

• The Data quality area, that includes a quality bar with three colors, and their respective percentages:

  • Invalid (red): Shows the percentage of values in the dataset that are considered invalid.

  • Empty (black): Indicates the percentage of values that are empty.

  • Valid (green): Displays the percentage of valid values in the dataset. The percentage does not take empty values into account.

• The Schema area that shows the different fields of the dataset, which data type or semantic type has been applied, and a quality bar for each field of the dataset.

Semantic types discovery

Each column of a dataset is automatically assigned a semantic type to better describe its content. Behind the scenes a data discovery operation occurs to determine which type to assign.