tSemanticMatch

Schema and Edit Schema

• A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

  Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion.

    If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the Repository Content window.

  This component offers the advantage of the dynamic schema feature. This allows you to retrieve unknown columns from source files or to copy batches of columns from a source without mapping each column individually. For further information about dynamic schemas, see Dynamic schema.

  This dynamic schema feature is designed for the purpose of retrieving unknown columns of a table and is recommended to be used for this purpose only; it is not recommended for the use of creating tables.

• Built-In: You create and store the schema locally for this component only.

• Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs.

The input column used as the query vector for searching. Select a column containing the data you want to match against (e.g., product description, customer feedback). This column's vector representation must be compatible with the vector field.

Vector DB

Select the vector database.

URI

Enter the endpoint URI in double quotes. For example, http://milvus-server:19530 or https://cloud-milvus.example.com.

Token

Enter the authentication token or API key required by your database. Leave blank if it runs without authentication.

Database name

Enter the name of the database containing your vector collection.

Collection name

Select the collection that stores the vector embeddings and associated data.

Vector field name

Select the field in the collection storing vector embeddings.

Text field name

This option is optional.

Select the field in the collection containing human-readable text or content. If specified, this text is returned in the Result output column. If not specified, only the score is returned.

Max results

Enter the maximum number of matching records to return per input record. Set to 1 for single match. Increase for multiple matches.

Min score

Enter the minimum similarity score threshold. The value must be in the range 0.0-1.0.

Records with scores below this value are returned in the reject output. For example, 0.6 means only records 60% similar or better are accepted.

tStatCatcher Statistics

Select this checkbox to gather the Job processing metadata at the Job level as well as at each component level.

Primary key

Metadata field name

Define filter conditions with an operator and a value. The available operators depend on the vector database, not Talend Studio.

The IN operator lets you define the exact values your data must match. For example: price IN 750,1000,1500 means that the value of the field price must be 750, 1000, or 1500. Any other value will be rejected.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error checkbox is cleared, if the component has this checkbox.