Data Errors
Click the Data Errors sub-tab and then click Change to Task Policy.
Data truncation errors
-
For a data truncation error: Click the triangle to open the list and select what happens when an truncation occurs in one or more specific records. You can select one of the following from the list:
- Ignore record: The task continues and the error is ignored.
- Log record to the exceptions table (default): The task continues and the error is written to the exceptions table.
- Suspend table: The task continues, but data from the table with the error record is moved into an error state and its data is not replicated
- Stop task: The task is stopped and manual intervention is required.
Data truncation error handling is supported with the following target endpoints only:
- MySQL
- Microsoft Azure Database for MySQL
- Google Cloud SQL for MySQL
- PostgreSQL
- Microsoft Azure Database for PostgreSQL
- Google Cloud SQL for PostgreSQL
- Google Cloud AlloyDB for PostgreSQL
- Microsoft Azure SQL Database
- Microsoft SQL Server
- Google Cloud SQL for SQL Server
- Microsoft Fabric Data Warehouse
- Oracle
- SAP Sybase ASE
- File
- Amazon Redshift
- Databricks Lakehouse (Delta)
- Databricks (Cloud Storage)
NOT NULL constraint violations
For source tables, Replicate can explicitly check for NULL values in each data column of each record, and whether the same column in the corresponding target table is marked as NOT NULL. In such cases, applying the record to the target will trigger a NOT NULL constraint violation that will be difficult to recover from if the task is running in Batch Optimized Apply mode. Therefore, the record is not applied to the target. Instead, the record is handled as an error (see below) without affecting other records in the batch being applied. While this way of handling NOT NULL constraint violation is very effective, it can impact performance, and if the target database does not actually enforce NOT NULL constraint violations, Replicate can be configured to skip that check in order to save CPU time and improve performance. Because Replicate generally knows what target databases enforce NOT NULL constraint violations, it is safe to always let Replicate activate this protection based on the target endpoint type. Still, Replicate gives you the option to override this default and either force a check (with its slight overhead) or disable the check and risk NOT NULL constraint violations.
-
Endpoint-determined: This means that the endpoint type (which can be an OLTP database or an analytical data warehouse) determines whether Replicate checks for NOT NULL constraint violations.
Information noteSome target endpoints do not support checking for NOT NULL constraint violations. In such cases, the tooltip next to the Endpoint-determined option will indicate that the current target endpoint does not allow checking for NOT NULL constraints.For OLTP-based target endpoints, the default is not to check for NOT NULL constraint violations as it is less critical (from a cost perspective) if a NOT NULL constraint violation occurs during the task. However, if a NOT NULL constraint violation occurs on any of the tables during the task, Replicate will switch to one-by-one mode, which will impact performance to a certain degree (depending on the number and size of the tables being replicated).
You can override the endpoint-determined handling by selecting either the Check or Don't check options described below.
Information noteThe string (Check) or (Don't check) will appear after Endpoint-determined according to the selected target endpoint type. -
Check: Select this if you always want Replicate to check for NOT NULL constraint violations. This might be useful if you know beforehand that some of the source table columns contain NULL values and the corresponding target table columns are configured with NOT NULL constraints. In this case, Replicate will not switch to one-by-one mode, but will continue change processing in Batch Optimized Apply mode. If a source table that will violate a NOT NULL constraint is detected during the check, Replicate will take the action described below. The default action is to suspend the table.
-
Don't check: Select this if you never want Replicate to check for NOT NULL constraint violations. This is recommended if you are confident that no such violations will occur during the task or if you do not mind Replicate switching to one-by-one mode if a NOT NULL constraint violation occurs.
- Suspend table: This is the default. The task continues, but tables with NULL values are suspended and their data is not replicated.
- Ignore record: The task continues and the error is ignored.
- Log record to the exceptions table: The task continues, but records with NULL values are written to the exceptions table.
- Stop task: The task is stopped and manual intervention is required.
For NOT NULL constraint violations, select one of the following:
Handling options when a NOT NULL constraint violation is discovered during a check:
For the Endpoint-determined and Check options, you can select how Replicate will handle the NOT NULL constraint violation:
Other data errors
-
For other data errors: Click the triangle to open the list and select what happens when an error occurs in one or more specific records. You can select one of the following from the list:
- Ignore record: The task continues and the error is ignored.
- Log record to the exceptions table (default): The task continues and the error is written to the exceptions table.
- Suspend table: The task continues, but data from the table with the error record is moved into an error state and its data is not replicated
- Stop task: The task is stopped and manual intervention is required.
Escalating error handling
-
Escalate error handling when other data errors reach (per table): Select this check box to escalate error handling when the number of non-truncation data errors (per table) reaches the specified amount. Valid values are 1-10,000.
Information noteThe escalation options are not available when the Apply changes using SQL MERGE task setting is enabled.
-
Escalation action: Choose what action Replicate should perform when error handling is escalated. Note that the available actions are dependent on the action selected from the For other data errors drop-down list described above.
- Log record to the exceptions table: The task continues, but the record with the error is written to the exceptions table.
-
Suspend table (default): The task continues, but data from the table with the error record is moved into an error state and its data is not replicated.
Information noteThe behavior differs according to the Change Processing Mode:
-
In Transactional apply mode, the last changes will not be replicated
-
In Batch optimized apply mode, a situation is possible where there will be no replication of data or data replication will occur in part
-
- Stop task: The task is stopped and manual intervention is required.
-