tDQRules / tDataQualityRules
Validates data using data quality rules by connecting Talend Studio to Qlik Talend Data Integration, or to a Talend app.
- Qlik Talend Data Integration.
- Talend Cloud Data Inventory.
- Talend Cloud Data Stewardship.
- The hybrid version of Talend Data Stewardship 8.0 R2022-07 and greater.
For Apache Spark Jobs, you can only use tDataQualityRules with the Talend apps.
For more information on each app, see the corresponding documentation:
- For Qlik Talend Data Integration, see Working with validation rules in Qlik Cloud documentation.
- For Talend Cloud Data Stewardship, see What is a data quality rule? in the Talend Cloud Data Stewardship User Guide.
- For the hybrid version of Talend Data Stewardship 8.0 R2022-07 and greater, see the What is a data quality rule? in the Talend Data Stewardship User Guide. For more information on the hybrid version, see What is hybrid for Talend Cloud?.
- For Talend Cloud Data Inventory, see the What is a data quality rule? in the Talend Cloud Data Inventory User Guide. You can connect to this app in Talend Studio 8.0 R2023-06 and greater.
A data quality rule is a set of business requirements which defines the values your data must comply with.
To retrieve the data quality rules into a JAR file, tDQRules connects Talend Studio to one of the app listed above. Talend Studio uses the retrieved library so you can apply data quality rules to your data. For more information on how the component works with Talend apps, see Description of the local and Cloud/hybrid process.
The following table describes the categories and subcategories into which the component returns the data:
Output categories | tDataQualityRules | tDQRules |
---|---|---|
Valid These data follow the Main flow. |
|
The data fulfill the condition and the validation expression or the data fulfill the alternative validation expression only. |
Invalid These data follow the Reject flow. |
|
. |
Not Applicable These data follow the Not_Applicable flow. |
Not applicable to this component | The data do not fulfill the condition. The rule cannot be applied to the data. |
This component is not shipped with your Talend Studio by default. You need to install it using the Feature Manager. For more information, see Installing features using the Feature Manager.
You need the Qlik Talend Cloud Enterprise Edition to use the tDQRules / tDataQualityRules component.
- In local mode, Apache Spark 3.0 and greater.
- Cloudera Data Engineering service with Apache Spark 3.1 or 3.2.
Standard properties
These properties are used to configure tDQRules and tDataQualityRules running in the standard Job framework.
The standard tDQRules and tDataQualityRules components belong to the Data Quality family.
If you are using tDataQualityRules, see Basic settings for tDataQualityRules . This component has been combined with tDQRules from Talend Studio 8.0 R2025-06 but the properties of tDataQualityRules are the same. They do not depend on the version you are using.
Basic settings for tDQRules
Properties | Description |
---|---|
Cloud | Select the app from which you want to retrieve the rules:
Click Apply to enable more settings. If you selected Talend Cloud Data Inventory or Stewardship, you are using tDataQualityRules, see Basic settings for tDataQualityRules . |
Property Type |
Select the way the file path and the schema will be set.
|
Schema and Edit schema |
A schema is a row description that defines the number of fields (columns) to be processed and passed on to the next component. Click Sync columns to retrieve the schema from the previous component connected in the Job. Select the Schema type:
Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. The supported types are: Boolean, Date, Double, Dynamic, Float, Integer, Long, Short, and String. |
URL |
Information noteImportant: You need the Rules - View
permission to retrieve the rules.
Enter the URL of the app. For example, https://qcs.us.qlikcloud.com/. |
API key | Enter your API key. To generate one, see Generating an API key. |
Rule library timestamp | After you entered the URL and token, click Fetch
rules. If the URL and token are correct, the rules are retrieved into Talend Studio: yyyy-MM-dd hh:mm:ss (library_number). When you click Fetch rules, a
JAR file containing the rule definition is
created
in:
When you update the rules in the app, click Fetch rules to retrieve the latest version. |
Rule name |
Select the rule from the list and click Refresh table. |
Rule content |
Associate the variables of the rule with the input data. The rule
is retrieved from the library:
When no rules or input columns are available, verify that:
|
Basic settings for tDataQualityRules
Properties | Description |
---|---|
Cloud |
This option is available from Talend Studio 8.0 R2025-06. Select the app from which you want to retrieve the rules:
Click Apply to enable more settings. If you selected Qlik Talend Cloud Data Integration, you are using tDQRules, see Basic settings for tDQRules . |
Schema and Edit schema |
A schema is a row description that defines the number of fields (columns) to be processed and passed on to the next component. Click Sync columns to retrieve the schema from the previous component connected in the Job. Select the Schema type:
Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. The supported types are: Boolean, Date, Double, Dynamic, Float, Integer, Long, Short, and String. |
Application | Select the application from which you want to retrieve the data quality rules. |
URL |
Information noteImportant: You need the Rules - View
permission to retrieve the rules.
Enter the URL of the selected app. For example, https://qcs.us.qlikcloud.com/ for Qlik Talend Data Integration. For Talend apps, the following URLs are supported:
|
Token | Enter your token. To generate one, see Generating a Personal Access Token. |
DQ rule library timestamp | After you entered the URL and token, click
Refresh. If the URL and token are correct, the rules are retrieved into Talend Studio: yyyy-MM-dd hh:mm:ss (library_number). When you click Fetch rules, a
JAR file containing the rule definition is
created
in:
When you update the rules in the Cloud or hybrid application, click Refresh to retrieve the latest version. |
Configure DQ rules | Associate the variables of the rule with the input data. The rules are
retrieved from the library:
If no rules or input columns are available, verify that:
You can associate the data types from Talend Studio with some data types from the apps. See Associating data types below. To apply more rules, click |
Associating data types
Rule variable from the app* | Input column (from Talend Studio) |
---|---|
Number | Double, Float, Integer, Long, Short, and String |
Boolean | Boolean |
Text | String |
Date | Date |
Any data types supported in the application | Dynamic |
* You can enter the URL of:
|
Advanced Settings
Properties | Description |
---|---|
tStatCatcher statistics |
Select this check box to gather the Job processing metadata at the Job level as well as at each component level. |