Data Profiling Attributes
Data profiling attributes can be used in the SELECT list, FROM clause, WHERE clause and ORDER BY clause of a MQL Query.
The following Data Profiling attributes are supported on file fields or table/view columns.
Data Profiling attribute name | Description | Value examples | Value data type | Supported operators in WHERE |
"Data Profiling".Blank | Number of blank value rows | 20 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Distinct" | Number of distinct rows | 1001 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Duplicate | Number of duplicate rows | 5 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Valid | Number of valid rows | 5 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Empty (Deprecated) | Number of empty rows: null in database or empty in files. This attribute is split into 3 attributes: "Data Profiling".Null, "Data Profiling".Blank and "Data Profiling".Zero. | 5 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Invalid | Number of invalid rows | 5 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Min | Lowest value | 1.0 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Max | Highest value | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Mean | Mean value | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Null | Number of null value rows | 10 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Variance | Variance value | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling".Median | Median value | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Lower Quantile" | Lower quantile | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Upper Quantile" | Upper quantile | 3.123456E30 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Avg Length" | Average length of strings | 6.5 | Double | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Min Length" | Minimum length of strings | 3 | Integer | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Max Length" | Maximum length of strings | 20 | Integer | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Inferred Data Types" | List of data types detected by the profiler | ["integer", "string"] | Array of String | =, !=, = ANY (...), != ANY (...), = ALL (...), != ALL (...), [NOT] EXISTS |
"Data Profiling".Zero | Number of zero value rows | 40 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
The following Data Profiling attributes are supported on files, tables and views.
Data Profiling attribute name | Description | Value examples | Value data type | Supported operators in WHERE |
"Data Profiling"."Profiled Date" | Last profiled date | 2022-09-24 13:25:17 | String | =, BEFORE, AFTER, BETWEEN ... AND ... |
"Data Profiling"."Processed Records" | Processed row count | 10 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
"Data Profiling"."Total Records" | Total row count | 100 | Long | =, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
When somebody uses a data profiling attribute in a filter, MIMM removes all objects from the results which are marked as hide data for that user. The hide data is controlled by multiple factors: whether the user has the Data Viewer capability object role assignment, whether a harvested object has a proposed or assigned data class resulting in a Sensitivity Label that has the HIDE DATA flag, or whether the object has a Sensitivity Label that has the HIDE DATA flag.