# tPostgresqlCDC

Addresses data extraction and transportation needs, only extracts the changes made to the source operational data and makes them available to the target system(s) using database CDC views.

The tPostgresqlCDC extracts source system data that has changed since the last extraction and transports it to another/other system(s).

Information noteNote: If batch processing is enabled in the Job, the component cannot capture the order of changes and the source and target databases will not be synchronized.

## tPostgresqlCDC Standard properties

These properties are used to configure tPostgresqlCDC running in the Standard Job framework.

The Standard tPostgresqlCDC component belongs to the Databases family.

The component in this framework is available in all subscription-based Talend products.

This component supports PostgreSQL vector data types. Use the VECTOR or Dynamic DB type in your schema column definition to store and then query vector embeddings. For more information about pgvector, read [the Amazon documentation](https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/).

## Basic settings

| Properties | Description |
| --- | --- |
| Database | Select the desired database type from the list and click<br>Apply. |
| Property type | Either Built-in or<br>Repository<br><br><ul> <li> <p> <span>Built-in</span>: No property data stored                           centrally.</p> </li> <li> <p> <span>Repository</span>: Select the repository                           file in which the properties are stored. The fields that follow are                           completed automatically using the data retrieved.</p> <div><span>Information note</span><span>Warning:</span> Reset the DB type by clicking the                           relevant button to select the CDC connection. </div> </li> </ul> |
| Use an existing<br>connection | Select this check box and in the Component<br>List drop-down list, select the desired connection component to<br>reuse the connection details you already defined.<br><br>Information noteNote: When a Job contains the parent Job and the child Job, do the following if you<br>want to share an existing connection between the parent Job and the child Job (for example,<br>to share the connection created by the parent Job with the child Job).<br><br><ol> <li>In the parent level, register the database connection to be shared               in the <span>Basic settings</span> view of the               connection component which creates that very database connection. </li> <li>In the child level, use a dedicated connection component to read               that registered database connection. </li> </ol><br><br>For an example about how to share a database connection across Job levels,<br>see [Sharing a database connection](https://help.qlik.com/talend/r/en-US/7.3/studio-user-guide/sharing-database-connection). |
| DB Version | Select the database version in the dropdown list. |
| Host | Enter the database server IP address. |
| Port | Listening port number of DB server. |
| Database | Enter the name of the database. |
| Schema | Enter the name of the PostgreSQL schema. |
| Username and Password | Enter the database user authentication data.<br><br>To enter the password, click the [...] button<br>next to the password field, enter the password in double quotes in the pop-up dialog box,<br>and click OK to save the settings. |
| Schema using CDC and Edit Schema | A schema is a row description, it defines the number of fields to be<br>processed and passed on to the next component. The schema is either Built-in or stored remotely in the Repository<br><br>Click Edit<br>schema to make changes to the schema. If the current schema is of the Repository type, three options are available:<br><br><ul> <li> <p> <span>View schema</span>: choose this                option to view the schema only.</p> </li> <li> <p> <span>Change to built-in property</span>:                choose this option to change the schema to <span>Built-in</span> for local changes.</p> </li> <li> <p> <span>Update repository connection</span>:                choose this option to change the schema stored in the repository and decide whether                to propagate the changes to all the Jobs upon completion.</p> <p>If you just want to                propagate the changes to the current Job, you can select <span>No</span> upon completion and choose this schema metadata                again in the <span>Repository Content</span>                window.</p> </li> </ul><br><br><ul> <li> <p> <span>Built-In</span>: You create and store the schema locally for this component                only.</p> </li> <li> <p> <span>Repository</span>: You have already created the schema and stored it in the                Repository. You can reuse it in various projects and Job designs.</p> <div><span>Information note</span><span>Warning:</span> Reset the DB type by clicking the                           relevant button to select the schema of the CDC connection. </div> </li> </ul><br><br>This component supports PostgreSQL vector data types. Use the<br>VECTOR or Dynamic DB type in your schema<br>column definition to store and then query vector embeddings. For more information about<br>pgvector, read [the Amazon documentation](https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/). |
| Table | Select the source table from which any changes made to data are to be<br>captured. |
| Subscriber | Enter the name of the application that will use the change table. |
| Events to catch | Insert: Select this checkbox to catch<br>the data inserted in the change table since the last extracion.<br><br><br>Update: Select this checkbox to catch<br>the data updated in the change table since the last extracion.<br><br><br>Delete: Select this checkbox to catch<br>the data deleted in the change table since the last extracion. |

## Advanced settings

| Properties | Description |
| --- | --- |
| Additional JDBC Parameters | Specify additional JDBC parameters for the database connection created.<br><br>This property is not available when the Use an existing connection check box in the Basic<br>settings view is selected. |
| Keep data in CDC Table | Select this checkbox to retain changes in the CDC table, allowing one or<br>more target systems to consult the data even after ingestion. When enabled, this<br>option prevents deletion of the consulted CDC data; instead, it updates the value<br>of the field CDC\_STATE to 'x'. This mechanism<br>lets users recheck previously ingested data by filtering records where<br>CDC\_STATE = 'x' (for example, using a<br>tMysqlInput component), and manage deletion themselves if needed. This option is<br>especially useful for users who need to audit or revisit ingested data without<br>having to re-run Jobs or reverse-engineer the process to understand how the<br>checkbox affects CDC\_STATE. |
| Use cursor | Select this checkbox to specify the number of rows you want to work with at<br>at any given time. This option optimises performance. |
| Trim all the String/Char columns | Select this checkbox to delete any spaces found at the beginning or end of<br>fields, in all of the columns containing character strings. |
| Trim column | Remove leading and trailing whitespace from defined columns.<br><br>Information noteNote: Clear Trim all the String/Char<br>columns to enable Trim columns in this field. |
| tStatCatcher Statistics | Select this checkbox to collect the log data at component level. |
| Enable parallel execution | Select this check box to perform high-speed data processing, by treating<br>multiple data flows simultaneously. Note that this feature depends on the database or<br>the application ability to handle multiple inserts in parallel as well as the number of<br>CPU affected. In the Number of parallel executions<br>field, either:<br><br><ul> <li>Enter the number of parallel executions desired.</li> <li>Press <span>Ctrl+Space</span> and select               the appropriate context variable from the list. For further information about context               variables, see <a href="https://help.qlik.com/talend/r/en-US/8.0/studio-user-guide/contexts-and-variables" target="_blank">Using contexts and variables</a>. </li> </ul><br><br>Note that when parallel execution is enabled, it is not possible to use global<br>variables to retrieve return values in a subJob.<br><br><ul><li>						            The <span>Action on							table</span> field is not available with the							parallelization function. Therefore, you must use a <span>tCreateTable</span> component if you							want to create a table. 					          </li><li>						            When parallel execution is enabled, it is not							possible to use global variables to retrieve return values in a							subJob.					          </li> </ul> |

## Global Variables

| Variables | Description |
| --- | --- |
| Global Variables | NB\_LINE: the number<br>of rows read by an input component or transferred to an output component. This is an<br>After variable and it returns an integer.<br><br>To fill up a field or expression with a variable, press Ctrl+Space to access the variable list and choose the variable to use from<br>it.<br><br>For more information about variables, see [Using contexts and variables](https://help.qlik.com/talend/r/en-US/8.0/studio-user-guide/contexts-and-variables).<br><br>A Flow variable functions during the execution of a component<br>while an After variable functions after the execution of the component. |

## Usage

| Usage guidance | Description |
| --- | --- |
| **Usage rule** | This component is used as a start component. It requires an output component<br>and a row Main link.<br><br>This component supports PostgreSQL vector data<br>types. |