# Data generation functions You can generate output data different from the input data. | Function | Random generation | Consistent generation | Bijective generation | Input data validation | | --- | --- | --- | --- | --- | | Generate from pattern | Yes | No | No | No | | Generate Uuid | Yes | No | No | No | | Generate sequence | Yes | No | No | No | | Generate from file/list | Yes | Yes | No | No | ## Generate from pattern This function generates a value based on a user-defined pattern. This function only applies on Strings. | Option | Description | | --- | --- | | Extra parameter | This function requires an extra parameter.

The extra parameter is a pattern that
follows those rules:

A is replaced with a random Latin uppercase letter.
a is replaced with a random Latin lowercase letter.
9 is replaced with a random digit.
H is replaced with a random Hiragana character.
K is replaced with a random full-width Katakana character.
k is replaced with a random half-width Katakana character.
C is replaced with a random Kanji character.
G is replaced with a random Hangul character.

All other characters are copied to the generated value
as is.

For more information about the supported
character types and the related Unicode ranges, see [Data masking functions in the masking components](https://help.qlik.com/talend/en-US/components/8.0/data-privacy/data-masking-functions-in-masking-components).

You can also use
numbered backreferences (\\<number>) using the following syntax: <pattern>\\<number>,<group1>,<groupN>.

<pattern> corresponds to the pattern to be used for generating the output value.
\\<number> is a numbered backreference. <number> identifies the position of the group placed after the "," character.
<group1>,<groupN> are comma-separated groups of characters. Each group is treated as a single unit. If a backreference calls a group, it is added as is in the generated value.

If you want to copy a character used in patterns
(A, a,
9, H,
h, K,
k, C,
G) as is in the generated value, use a
backreference. | This function does not work correctly if a comma ',' is used in the pattern. In the following example: * a characters are replaced with random Latin lowercase letters. * s characters are not masked in the generated output. * \\2 calls the group placed after the second "," character, which is @talend.com. | Input value | Extra parameter | Example of a masked value | | --- | --- | --- | | A26 | "aaaass\\2,@gmail.com,@talend.com" | hjdfss@talend.com | In the following example: * \\3 calls the group placed after the third "," character, which is a. * 9 characters are masked with random digits. | Input value | Extra parameter | Example of a masked value | | --- | --- | --- | | A26 | "\\39999,D,Z,a" | a4825 | ## Generate UUID This function masks the input value with a randomly generated universally unique identifier (UUID). This function uses the UUID.randomUUID() method provided by Java. This Java method does not use a seed, meaning that if you run the job twice, the function generates different UUIDs. This function is applied on Strings. This function requires no extra parameter. In the following example, the masked valued is a randomly generated UUID. | Input value | Example of a masked value | | --- | --- | | A26 | 28e92000-aafa-4ec3-bd56-240f192a4a8c | ## Generate sequence This function returns the extra parameter, and, for each row, will increase this number by 1. This function can be applied on all data types but Date (Integer, Long, Strings, etc.). Information noteNote: This function is not supported in the Spark version of the component. | Option | Description | | --- | --- | | Extra parameter | This function requires an extra parameter.

The extra parameter must be a number.

If the extra
parameter is not a number, it is set to 0. | In the following example, the generated sequence starts with the number set as an extra parameter. | Input values | Extra parameter | Examples of masked values | | --- | --- | --- | | 21

A48 | "0" | 0

1 | ## Generate from file/list This function randomly replaces the input value with one of the user-defined values. This function is applied to Strings or numerical data types. | Option | Description | | --- | --- | | Method | The Randomly method randomly selects the value
from the list (or file). As a result, two similar input values can be masked
with the different output values.

The Consistently
method ensures that two similar input values are masked with the same output
value.

When using the
Consistently method, the probability of
generating duplicates can be calculated using the following formulas:

P = 1 if K < N, or
P = 1-K*(K-1)*(K-2)*…*(K-N+1) / K^N

where P
is the probability of generating duplicates, N the input data size and K is the size of the input list given as a
parameter.

Using this approach, it is possible to
calculate the probability to find a pair sharing the same value within a
group.

For example, the probability that, in a group
of n people, two people have the same birthday is
the following:

2.7% in a group of 5 people,
41.1% in a group of 20 people,
100% in a group of 367 people, since there are 366 possible birthdays, including February 29.

| | Extra parameter | This function requires an extra parameter.

The extra parameter can be:

a comma-separated list of two values minimum; or
a path to a file containing the values.

The values must be stored in a String and
separated by commas, for example: "item1, item2,
item3, etc.". This function uses the hashCode() method provided by Java to choose an element from
the list.

If you use the Apache Spark version of the component, set the file
path as follows:

In local mode:
- Apache Spark 3.1 and earlier: prefix://file path or file:///file path.
- Apache Spark 3.2 and later: file:///file path.
In Standalone and Yarn modes: prefix://file path.
If the index is on a cluster: hdfs://hdpnameservice1/file path.

Paths to folders are not supported.

If the extra parameter is not set, the function returns an empty String or
0. | In the following example, the masked value is one of the values set as extra parameters. | Input value | Method | Extra parameter | Examples of a masked value | | --- | --- | --- | --- | | 21 | Randomly | "help,documentation" | help |