# tDuplicateRow

Creates duplicates with meaningful data for data quality functional testing purposes.

tDuplicateRow generates duplicate data from an input flow. It groups similar duplicates together and identifies the original record of each group by true.

This component can be used in combination with the tRowGenerator component to generate duplicate data.

This component is not shipped with your Talend Studio by default. You need to install it using the Feature Manager. For more information, see [Installing features using the Feature Manager](https://help.qlik.com/talend/r/en-US/8.0/studio-user-guide/install-features-to-talend-studio).

## tDuplicateRow Standard properties

These properties are used to configure tDuplicateRow running in the Standard Job framework.

The Standard tDuplicateRow component belongs to the Data Quality family.

This component is available in Talend Data Management Platform, Talend Big Data Platform, Talend Real-Time Big Data Platform, Talend Data Services Platform and in Talend Data Fabric.

## Basic settings

| Properties | Description |
| --- | --- |
| Schema and Edit schema | <ul> <li> <p>A schema is a row description. It defines the number of fields            (columns) to be processed and passed on to the next component. When you create a Spark            Job, avoid the reserved word <kbd>line</kbd> when naming the            fields.</p> <p>Click <span>Sync columns</span>                            to retrieve the schema from the previous component in the Job.</p> <p>The output schema of this component contains one read-only                            column, <em>ORIGINAL_MARK</em>. This column identifies by                                <span>true</span> or <span>false</span> if the record is an original record or a duplicate                            record respectively. There is only one original record per group.</p> </li> <li> <p> <span>Built-In</span>: You create and store the schema locally for this component                only.</p> </li> <li> <p> <span>Repository</span>: You have already created the schema and stored it in the                Repository. You can reuse it in various projects and Job designs.</p> </li> </ul> |
| Percentage of duplicated records | Enter the percentage of the duplicate rows you want to<br>have in the output flow. |
| Distribution of duplicates | Name: Select the probability<br>distribution you want to use to generate duplicates: Bernoulli<br>distribution or Poisson distribution or Geometric distribution.<br><br><br>Average group size: Set the<br>average number of duplicate records you want to generate in the groups<br>of duplicates. |
| Modifications | Define in the table what fields to change in a row and<br>how to change them:<br><br>-Input Column:<br>Select the column from the input flow from which you want to generate<br>duplicates through modifying its values.<br><br>-Modification<br>Rate: Enter a rate of the modifications you want to have in<br>the duplicate record generated from an input column. The rate is a value<br>between *0* and *1*. If you<br>set the rate to *0*, no modification will be done.<br>If you set the rate to *0.5*, modification will be<br>done in average every two rows. But if you set the rate to *1*, modification is done at each row.<br><br>These modifications are based on the function you select<br>in the Function column and the<br>number of modifications you set in the Max<br>Modification Count column.<br><br>-Function: Select<br>the function that will decide what modification to do on a value to<br>duplicate it. For example, you can decide to have exact or approximate<br>duplicate values through replacing or adding letters or numbers,<br>replacing values with synonyms from an index file or deleting values by<br>setting the function to null or blank.<br><br>The Function list<br>will vary according to the column type. For example, a column of a *String* type will have an *Add<br>letters* option in the list while a column of an *Integer* type will have an *Add<br>digits* option in the list. Also, the Function list for a *Date* column<br>is date-specific. For further information about the functions used on *Date* columns, see [Date functions in tDuplicateRow](https://help.qlik.com/talend/en-US/components/8.0/data-privacy/tduplicaterow-tduplicaterow-date-functions-in-tduplicaterow-standard-component-enterprise-the).<br><br>-Max Modification<br>Count: Enter a maximum number of the values to be<br>modified.<br><br>-Synonym Index<br>Path: Set the path to the index file from which you use<br>the synonym.<br><br>This field is available if you select the *Synonym replace* function which means that the<br>value in the duplicate record is replaced by one of its synonym,<br>according to the given the rate.<br><br>You must use the tSynonymOutput component to create a Lucene index and<br>feed it with synonyms. For further information about how to create a<br>synonym index and define the reference entries, see [tSynonymOutput](https://help.qlik.com/talend/en-US/components/8.0/synonym-index/tsynonymoutput). |

## Advanced settings

| Properties | Description |
| --- | --- |
| Seed for random generator | Set a random number if you want to generate the same<br>sample of duplicates in each execution of the Job.<br><br>Repeating the execution with a different value for the<br>seed will result in a different duplicate sample being generated.<br><br>Keep this field empty if you want to generate a different<br>duplicate sample each time you execute the Job. |
| tStat<br>Catcher<br>Statistics | Select this check box to collect log data at the<br>component level. |

## Usage

| Usage guidance | Description |
| --- | --- |
| **Usage rule** | This component helps you to generate duplicate data of an<br>input flow according to certain criteria and use it for testing<br>purposes. |

**Related tasks**

* [Generating duplicate data from an input flow](https://help.qlik.com/talend/en-US/components/8.0/data-privacy/generating-duplicate-data-from-input-flow "This scenario describes a basic Job that generates a sample of duplicate data from an input flow by using probability theories and specific criteria on three columns: Name, City and DOB (date of birth).")