# tPatternMasking Standard properties

These properties are used to configure tPatternMasking running in the Standard Job framework.

The Standard tPatternMasking component belongs to the Data Quality family.

## Basic settings

| Properties | Description |
| --- | --- |
| Schema and Edit Schema | <ul> <li> <p>A schema is a row description. It defines the number of fields            (columns) to be processed and passed on to the next component. When you create a Spark            Job, avoid the reserved word <kbd>line</kbd> when naming the            fields.</p> <p>Click <span>Sync        columns</span> to retrieve the schema from the previous component connected in the      Job.</p> <p>Click <span>Edit        schema</span> to make changes to the schema. If the current schema is of the <span>Repository</span> type, three options are available:</p> <ul> <li> <p> <span>View schema</span>: choose this                option to view the schema only.</p> </li> <li> <p> <span>Change to built-in property</span>:                choose this option to change the schema to <span>Built-in</span> for local changes.</p> </li> <li> <p> <span>Update repository connection</span>:                choose this option to change the schema stored in the repository and decide whether                to propagate the changes to all the Jobs upon completion.</p> <p>If you just want to                propagate the changes to the current Job, you can select <span>No</span> upon completion and choose this schema metadata                again in the <span>Repository Content</span>                window.</p> </li> </ul> <div>The output schema of this component contains        read-only columns: <ul> <li> <span>TWEAK</span>: Is generated when the              <span>Use tweaks with FF1 Encryption</span> check box is            selected. This column contains the tweak necessary to decrypt the data.</li> <li> <span>ORIGINAL_MARK</span>: Identifies by              <span>true</span> or <span>false</span> if the record is an original record or a substitute            record respectively.</li> </ul> </div> </li> <li> <p> <span>Built-In</span>: You create and store the schema locally for this component                only.</p> </li> <li> <p> <span>Repository</span>: You have already created the schema and stored it in the                Repository. You can reuse it in various projects and Job designs.</p> </li> </ul> |
| Modifications | Define in the table what fields to change and how to change<br>them:<br><br><br>Column to mask: Select the column from the input<br>flow for which you want to generate similar data by modifying its<br>values.<br><br>You can mask data from different columns but you need to follow the order<br>of the fields you want to mask.<br><br>Each column is processed sequentially, meaning that data masking<br>operations will be performed on the data from the first column, the<br>second column, and so on.<br><br>In a colum, each data field is a fixed length field, except the last data<br>field.<br><br>For fixed length fields, each value must contain the same number of<br>characters, for example: "30001,30002,30003" or<br>"FR,EN".<br><br>In a column, the last Enumeration or Enumeration<br>from file data field is a variable length field.<br><br>For variable length fields, each value might not always contain the same<br>number of characters, for example:<br>"30001,300023,30003" or<br>"FR,ENG".<br><br>Field type: Select the field type the data belongs to.<br><br><ul> <li> <p> <span>Interval</span>: When selected, set a range                                        of numeric values used for masking purposes in the                                            <span>Range</span> field, using the following                                        syntax: <kbd>"&lt;min&gt;,&lt;max&gt;"</kbd>.</p> <p>The number of masked characters from the input data                                        corresponds to the number of characters of the maximum                                        value.</p> <p>For example, <kbd>"1,999"</kbd> will be                                        interpreted as <kbd>"001,999"</kbd>, which means                                        that three characters from the input data will be masked by                                        a value randomly selected from the defined range of                                        values.</p> </li> <li> <p> <span>Enumeration</span>: When selected, enter a                                        comma-separated list of values to be used for masking data                                        in the <span>Values</span> field, using the                                        following syntax:                                            <kbd>"value1,value2,value3"</kbd>.</p> </li> <li> <span>Enumeration from                                        file</span>: When selected, set the path to the CSV                                    file containing a list of values to be used for masking data in                                    the <span>Path</span> field. The file                                    must contain one value per row and each value must be                                    unique.</li> <li> <p> <span>Date pattern (YYYYMMDD)</span>: When                                        selected, set a range of years in the <span>Date                                            Range</span> field, using the following syntax:                                            <kbd>"&lt;min_year&gt;,&lt;max_year&gt;"</kbd>.</p> <p>Years can only have four digits, for example:                                            <kbd>"1900,2100"</kbd>.</p> <p>The input dates to be masked must follow the YYYYMMDD                                        pattern, for example: <kbd>20180101</kbd>.</p> <p>For example, if the input date is                                            <kbd>20180101</kbd> and the value in the                                            <span>Date Range</span> is                                            <kbd>"1900,2100"</kbd>,                                            <span>19221221</span> could be the output date.</p> </li> </ul><br><br>In the Values, Path,<br>Range and Date Range,<br>values must be enclosed in double quotes.<br><br>When the input data is invalid, meaning that a value<br>does not match the pattern defined in the component, the generated value is<br>null. |

## Advanced settings

| Properties | Description |
| --- | --- |
| Method | The component uses Format-Preserving Encryption (FPE)<br>methods to generate masked output values in the same format as the input values.<br><br>The FPE methods are bijective methods, except when<br>using tweaks.<br><br>The Basic method is the default<br>algorithm.<br><br>Information noteNote: As the masking methods are stronger, it is recommended to use the FF1<br>algorithms rather than the Basic method.<br><br>The FF1 with AES method is based<br>on the Advanced Encryption Standard in CBC mode. The FF1 with<br>SHA-2 method depends on the secure hash function HMAC-256.<br><br>You can use those methods only if the number of possible<br>values the component can generate from the input pattern is greater than<br>or equal to 1,000,000.<br><br>Information noteNote: Java 8u161 is the minimum<br>required version to use the FF1 with AES method.<br>To be able to use this FPE method with Java versions earlier than 8u161, download the<br>Java Cryptography Extension (JCE) unlimited strength jurisdiction policy files from<br>Oracle website.<br><br>The FF1 with AES and<br>FF1 with SHA-2 methods require a password to<br>be specified in the Password or 256-bit key for FF1 methods<br>field of the Advanced settings to generate unique<br>masked values. |
| FF1 settings | Password or 256-bit key for FF1 methods: Set the password or secret key required for the<br>FF1 with AES and FF1<br>with SHA-2 methods to generate unique masked<br>values. If the password is not set, a random password is created<br>at each Job execution. When using the FF1 with<br>AES and FF1 with SHA-2<br>methods and a password, the seed from the Seed for<br>random generator field is not<br>used.<br><br><br>You can get the 256-bit key using:<br><br><ul> <li>Online tools</li> <li>OpenSSL: <span>openssl rand -base64 32</span> </li> </ul><br><br><br>Use tweaks with FF1 Encryption: Select this check box to use tweaks. A unique tweak is<br>generated for each record and applies to all data of a record.<br>If bijective masking is necessary, do not use this feature.<br>For more information about tweaks, see the [data masking functions](https://help.qlik.com/talend/r/en-US/8.0/components/data-privacy/data-masking-functions-in-masking-components).<br><br><br>Use a column containing the tweaks: Available when Use tweaks with FF1<br>Encryption check box is selected. Select this check box to<br>use an input column as the input for tweaks which must be 32 digit<br>hexadecimal strings.<br><br><br>Column containing the tweaks: Available when the Use a column<br>containing the tweaks check box is selected. Select the<br>column that contains the tweaks.<br><br><br><br>Key derivation function<br>: Select the key derivation function. Jobs created from<br>Talend Studio 8.0 R2022-04 run using PBKDF2 with 300,000<br>iterations.<br><br>When you import a Job prior to Talend Studio 8.0 R2022-04, you can run the Job using 300,000 iterations. The results will<br>be different than using 65,536 iterations. |
| Seed for random generator | Set a random number if you want to generate the same sample of<br>substitute data in each execution of the Job. The seed is not set by default.<br><br>If you do not set the seed, the component<br>creates a new random seed for each Job execution. Repeating the execution with a<br>different seed will result in a different sample being generated. |
| Encoding | Select the encoding from the list or select Custom and define it manually. If you select Custom and leave the field empty, the supported<br>encodings depend on the JVM that you are using. This field is compulsory for the file<br>encoding. |
| Output the original row | Select this check box to output original data rows in addition to the<br>substitute data. Outputting both the original and substitute data can be useful in debug<br>or test processes. |
| Null input returns null | This check box is selected by<br>default. When selected, the component outputs null when<br>input values are null.<br><br>When cleared, and when the input data is null, the input data<br>are sent to the "Invalid" output flow.<br><br>From Talend Studio<br>R2024-08 onwards, when Null input returns null is selected<br>and the input data is null, the masking function is not applied,<br>null is returned and the input data are sent to the main flow. |
| Empty input returns an empty output | When this check box is selected, empty values are left unchanged in<br>the output data. Otherwise, the selected functions are applied to the input<br>data. |
| Send invalid data to<br>"Invalid" output flow | This check box is selected by default.<br><br><ul> <li>Selected: When the data can be masked, they are sent to the                                    main flow. Otherwise, the data are sent to the "Invalid" output flow.</li> <li>Cleared: The data are sent to the main flow.</li> </ul><br><br><br>Invalid data are any values that do not match the pattern. |
| tStat Catcher Statistics | Select this checkbox to gather the Job processing metadata at the Job level<br>as well as at each component level. |

## Usage

| Usage guidance | Description |
| --- | --- |
| **Usage rule** | This component is an intermediary step. It requires an input and<br>output flows. |