Skip to main content Skip to complementary content

Matching two records

You can use the tMatchGroup component to detect duplicates and define how to merge similar records to create a master record.

Creating a master record is an iterative process: each new master record can be used to find new duplicates.

You can choose between two different algorithms to create master records:

  • Simple VSR Matcher
  • T-Swoosh. This algorithm is only available in the Standard component.

The main difference between the two algorithms is that T-Swoosh creates, for each master record, a new record that does not exist in the list of input records.

Matching measures

To compare one attribute of two records, you can use any of the implemented matching functions, such as Exact, Levenshtein and Jaro-Winkler, or a custom matching algorithm you created.

You can also compare two records on many attributes. For two records to match, the following two conditions must hold:

  • When using the T-Swoosh algorithm, the score for each matching function in the match rule must exceed the threshold, if any specified. By default, the threshold is set to 1. This means exact match for most matching functions, excepted for Exact - ignore case and potentially any custom matching function.
  • The global score, computed as a weighted score of the different matching functions, must exceed the match threshold. The score is equal to Σ(wi × si(r1,r2)) / Σwi where wi is the confidence weight of the matching function i and si(r1,r2) is the score of the matching function i over records r1 and r2 .
Configuration of the tMatchGroup component.

In this example, the score for the Jaro-Winkler metric on the fname attribute must exceed 0.7 and the global score, with a confidence weight of 1 on each of the two attributes, must exceed 0.85.

Example of the weighted average computation.
This example shows the weighted average computation that yields to the global score of two similar records:
  1. As the Confidence Weight of both attributes is set to 1, the normalized weight of each attribute is 0.5.
  2. The attribute matching distance is 1 for the lname attribute and 0.722... for the fname attribute.
  3. The score is calculated as follows: 0.5 x 1 + 0.5 x 0.722... = 0.8611...

Match rules

You can define different match rules for the same dataset.

Two records match if at least one of the match rules is satisfied. As soon as two records match according to a given rule, the other rules are not checked.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here