Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Adding a new regular expression-based semantic type through command line interface

Availability-noteDeprecated

You can create a semantic type based on a regular expression in Talend Dictionary Service and add it to the list of recognized data types in Talend Data Preparation.

In Talend Data Preparation, not every type of data can currently be matched with one of the predefined semantic types. Italian social security numbers, also known as codice fiscale, are currently not recognized for example.

Let's say that you work for an Italian company, only dealing with Italian customers. In this example, you need to clean some customer data, such as their names, email address, or their social security number. The semantic type for the column containing the social security number data will be set by default to text. This is a bit disappointing and you would like to create a more specific category in order to match this type of data: a codice_fiscale semantic type in this case.

You will create this new semantic type in Talend Dictionary Service, and it will be automatically available in Talend Data Preparation so that your data can be matched with a proper type.

Procedure

  1. Create a .txt file containing the following regular expression and save it as REGEX_CODICE_FISCALE.txt.

    This regular expression is designed to match the Italian codice fiscale, which is an alphanumeric code of 16 characters. Data that matches that pattern in Talend Data Preparation will be identified as codice fiscale.

  2. Add this file to the <Dictionary_Service_Path>/command-line/samples/source folder.

    This folder is used for the sake of this example, but you can save it to your preferred location.

  3. Open a command prompt window.
  4. Using the cd command, go to the <Dictionary_Service_Path>/command-line folder.
  5. To create the new codice_fiscale semantic type in Talend Dictionary Service and configure its different parameters, execute the following command according to your operating system:
    • category_manager.bat -c -name codice_fiscale -type REGEX -desc "Italian social security number" -src samples\source\REGEX_codice_fiscale.txt for Windows
    • ./category_manager.sh -c -name codice_fiscale -type REGEX -desc "Italian social security number" -src samples/source/REGEX_codice_fiscale.txt for Linux

    Please note that to be able to use this command, you need to put it on one single line.

    You are prompted for your Talend Administration Center credentials. The command is executed after you enter a valid login and password.

    The codice_fiscale semantic type is now added to the list of categories in Talend Dictionary Service.

  6. Go back to Talend Data Preparation and open your dataset with the column containing the social security numbers.

    The change in semantic types is instantly effective in Talend Data Preparation for every new dataset that you import. For existing datasets, you need to manually change the column type.

  7. To apply the new codice_fiscale semantic type to your column, click the white arrow next to the column name.
  8. Click This column is a... > codice_fiscale

    The column type now matches the newly created category.

Results

Your data is now matched with the codice_fiscale semantic type, that you manually created in Talend Dictionary Service. From now on, when importing new datasets containing Italian social security numbers, they will automatically be matched with the proper type.

To display a list of all the available commands in Talend Dictionary Service, go to <Dictionary_Service_Path>/command-line and enter the following command according to your operating system:

  • category_manager.bat -h command for Windows
  • ./category_manager.sh -h for Linux

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here