Indexing a reference data set in Elasticsearch
This scenario applies only to Talend Platform products with Big Data and Talend Data Fabric.
In this Job, the tMatchIndex component creates an index in Elasticsearch and populates it with a clean and deduplicated data set which contains a list of education centers in Chicago.
After performing all the matching actions on the data set which contains a list of education centers in Chicago, you do not need to restart the matching process from scratch when you get new data records having the same schema. You can index the clean data set in Elasticsearch using tMatchIndex for continuous matching purposes.
-
You generated a pairing model using tMatchPairing.
For further information, see Computing suspect pairs and writing a sample in and Computing suspect pairs and suspect sample from source data.
You can find examples of how to generate a pairing model on Talend Help Center (https://help.talend.com).
-
Make sure the input data you want to index is clean and deduplicated.
For an example of how to clean and deduplicate a data set, see Creating a clean data set from the suspect pairs labeled by tMatchPredict and the unique rows computed by tMatchPairing.
You can find an example of how to clean and deduplicate a data set on Talend Help Center (https://help.talend.com).
-
The Elasticsearch cluster must be running Elasticsearch 5+.