# tDataQualityRules properties for Apache Spark Streaming (deprecated)

Availability-note[Deprecated](https://help.talend.com/r/en-US/Cloud/labels-on-talend-help-center/availability-deprecated)

This content is deprecated.

These properties are used to configure tDataQualityRules running in the Spark Streaming Job framework.

Information noteDeprecation: Spark Streaming is deprecated beginning with the R2026-09 release. Use Spark Structured Streaming instead.

The Spark Streaming tDataQualityRules component belongs to the Data Quality family.

Information noteAttention: From Talend Studio 8.0 R2025-06, the tDataQualityRules Standard component is replaced by tDQRules. This component lets you use the rules from:

* Qlik Talend Data Integration.
* Talend Cloud Data Inventory.
* Talend Cloud Data Stewardship.
* The hybrid version of Talend Data Stewardship 8.0 R2022-07 and greater.

For Apache Spark Jobs, you can only use tDataQualityRules with the Talend apps.

## Basic settings

| Properties | Description |
| --- | --- |
| Output schema and Edit schema | A schema is a row description that defines the number of fields (columns) to be<br>processed and passed on to the next component.<br><br>Click Sync<br>columns to retrieve the schema from the previous component connected in the<br>Job.<br><br>Select the Schema type:<br><br><ul> <li> <p> <span>Built-In</span>: You create and store the schema locally for this component                only.</p> </li> <li> <p> <span>Repository</span>: You have already created the schema and stored it in the                Repository. You can reuse it in various projects and Job designs.</p> </li> </ul><br><br>Click Edit<br>schema to make changes to the schema. If the current schema is of the Repository type, three options are available:<br><br><br>View schema: choose this<br>option to view the schema only.<br><br><br>Change to built-in property:<br>choose this option to change the schema to Built-in for local changes.<br><br><br>Update repository connection:<br>choose this option to change the schema stored in the repository and decide whether<br>to propagate the changes to all the Jobs upon completion.<br><br>The supported types are: Boolean, Date, Double, Float, Integer, Long, Short,<br>and String. |
| Application | Select the application from which you want to retrieve the data quality<br>rules.<br><br>This option is available from Talend Studio 8.0 R2024-01 onwards. |
| URL | Information noteImportant: You need the Rules - View<br>permission to retrieve the rules. For more information, see the [predefined user roles](https://help.qlik.com/talend/r/en-US/Cloud/management-console-user-guide/tmc-predefined-roles) for the<br>app you are using.<br><br>Enter the URL of the selected app. When the URL<br>does not match the selected app, the Job may fail. The following URLs are<br>supported:<br><br><ul> <li><span>Talend Cloud Data Stewardship</span>, or the hybrid version of <span>Talend Data Stewardship</span> 8.0 R2022-07 and greater: <pre><code><pre><code><code>https://tds.&lt;env&gt;.cloud.talend.com/rulerepository/api/v1https://tds.&lt;env&gt;.cloud.talend.com/rulerepository/api/v1/https://tds.&lt;env&gt;.cloud.talend.com/rulerepositoryhttps://tds.&lt;env&gt;.cloud.talend.com/rulerepository/https://tds.&lt;env&gt;.cloud.talend.com (Only for <span>Talend Cloud Data Stewardship</span>)https://tds.&lt;env&gt;.cloud.talend.com/ (Only for <span>Talend Cloud Data Stewardship</span>)</code></code></pre></code></pre> <p>When you use the hybrid version, you can use a URL with the                                          IP address or the hostname:</p><div> <pre><code><pre><code><code>https://ip:19999/rulerepository/api/v1https://ip:19999/rulerepository/api/v1/https://ip:19999/rulerepositoryhttps://ip:19999/rulerepository/https://hostname:19999/rulerepository/api/v1https://hostname:19999/rulerepository/api/v1/         https://hostname:19999/rulerepositoryhttps://hostname:19999/rulerepository/</code></code></pre></code></pre> </div> </li> <li><span>Talend Cloud Data Inventory</span>, from <span>Talend Studio</span> 8.0                                    R2023-06:<pre><code><pre><code><code>https://tdc.&lt;env&gt;.cloud.talend.com/rulerepository/api/v1https://tdc.&lt;env&gt;.cloud.talend.com/rulerepository/api/v1/https://tdc.&lt;env&gt;.cloud.talend.com/rulerepositoryhttps://tdc.&lt;env&gt;.cloud.talend.com/rulerepository/https://tdc.&lt;env&gt;.cloud.talend.comhttps://tdc.&lt;env&gt;.cloud.talend.com/ </code></code></pre></code></pre></li> </ul><br><br><br>where &lt;env&gt; part is the name of your Cloud region. See [Accessing Talend Cloud applications](https://help.qlik.com/talend/en-US/installation-guide-windows/Cloud/talend-cloud-application-url). |
| Token | Enter your personal access token. To generate one, see [https://help.talend.com/r/en-US/Cloud/management-console-user-guide/cloud-access-token](https://help.qlik.com/talend/r/en-US/Cloud/management-console-user-guide/cloud-access-token). |
| DQ rule library timestamp | After you entered the URL and token, click<br>Refresh.<br><br>If the URL and token are correct, the data<br>quality rules are retrieved into Talend Studio: yyyy-MM-dd hh:mm:ss (library\_number).<br><br>When you click Refresh, a<br>JAR file containing the rule definition is<br>created<br>in:<br><br>```<br>studio-path\configuration\.m2\repository\org\talend\libraries\rule-number<br>```<br><br>When you update the data quality rules in the Cloud or hybrid<br>application, click Refresh to retrieve the latest<br>version. |
| Configure DQ rules | Associate the variables of the rule with the input data.<br><br>The rules are<br>retrieved from the library:<br><br><ul> <li><span>DQ Rule</span>: Select the rule.</li> <li><span>Rule variable</span>: The variables of the rule are                              automatically retrieved.</li> <li><span>Input column</span>: Select the column that contains                              the values that must replace the variable.</li> </ul><br><br>If no rules or input columns are available, verify that:<br><br><ul> <li>Data quality rules have been retrieved in <span>DQ rule library                                    timestamp</span>.</li> <li>The input schema is correct.</li> </ul><br><br>You can associate the data types from Talend Studio with some data types from the apps. See [Associating data types](https://help.qlik.com/talend/en-US/components/8.0/validation/tdqrules#associating-data-types) below.<br><br>To apply more rules, click ![Plus icon](https://help.qlik.com/talend/en-US/components/8.0/Resources/images/plus_button.png). |

## Associating data types

The following table describes the data types you can associate.

| Rule variable from the app\* | Input column (from Talend Studio) |
| --- | --- |
| Number | Double, Float, Integer, Long, Short, and String |
| Boolean | Boolean |
| Text | String |
| Date | Date |
| \* You can enter the URL of:<br><br><ul> <li><span>Talend Cloud Data Stewardship</span>.</li> <li>The hybrid version of <span>Talend Data Stewardship</span> 8.0 R2022-07 and greater.</li> <li><span>Talend Cloud Data Inventory</span>, from <span>Talend Studio</span> 8.0 R2023-06.</li> </ul> |  |

## Advanced settings

| Properties | Description |
| --- | --- |
| tStatCatcher<br>statistics | Select this checkbox to gather the Job processing metadata at the Job level<br>as well as at each component level. |