Setting advanced properties

In the Advanced tab, you can configure CDC-related options as well as any additional settings or internal parameters that may be required.

Select either Trigger-based CDC or Log-based CDC according to your preferred CDC method.

Trigger-Based CDC

• Trigger-based mode: The default is Commit Timestamp (CTS). This is the preferred method of working as it offers better performance. If you are configuring the SAP HANA source endpoint for the first time, there is no reason at all to choose Log Table mode. However, if you upgraded from a Replicate version configured with the Use log table option, Log Table mode will be automatically selected to prevent tasks from failing. You can then switch to Commit Timestamp (CTS) mode as described in Switching from Log Table mode to Commit Timestamp (CTS) mode below.

• Create CDC artifacts: When this option is enabled (the default), Replicate creates the required CDC artifacts in the source database. Clear this check box if you would rather create the CDC artifacts manually.

• Create CDC table in schema: The name of the schema in the source database where you want the attrep_cdc_changes table to be created.

  Information note

  The specified schema must already exist in the source database.

• Log cleanup interval (min): Specify how often (in minutes) to check if Change Records need to be deleted from the attrep_cdc_changestable.
• Log retention period (min): Specify how long (in minutes) to keep Change Records in the attrep_cdc_changes table before deleting them.
• Limit number of records in each retrieval to: When the value is set to "0", Replicate will retrieve all records with changes. To improve performance, you might want to limit the number of records retrieved in each CDC operation.

• SAP archiving users to filter: Specify the user names of SAP application users whose DELETE operations you do not want Replicate to capture. Users must be separated with commas.

  Information note

  Editing the user list requires you to stop and then resume the task for the changes to take affect.

Switching from Log Table mode to Commit Timestamp (CTS) mode

As Commit Timestamp (CTS) mode requires different database artifacts than Log Table mode, you need to create new database artifacts. These are as follows:

• The new Replicate CDC changes table
• The new source table capture triggers

To prevent data loss while switching modes, the new artifacts are created while the old ones are still active. You can create the new artifacts manually or automatically. If you have multiple tasks using the same SAP HANA source endpoint, you need to repeat the process for each task.

Creating the new artifacts automatically and switching to Commit Timestamp (CTS) mode

1. Stop the task using the SAP HANA endpoint in Log Table mode.
2. Select Commit Timestamp (CTS) from the Trigger-based mode drop-down list.

3. In the Advanced tab of the endpoint settings, make sure the Create CDC artifacts option is enabled.

   Now the endpoint is configured to create the artifacts automatically. 
   

4. Open the Advanced Run Options dialog, select Metadata only and then select Create missing tables and then stop.
5. Click OK and wait for the task to complete.
   
6. Verify that the following artifacts have been created in the SAP HANA database:
   • A new table called attrep_cdc_changes_cts.
   • All of the source tables have new triggers with "V3" in their name 
7. Delete the old triggers. Old triggers will not have "V3" in their names.
8. Select Log Table from the Trigger-based mode drop-down list.
9. Resume the task and wait for it to complete capturing changes. You can compare the stream position in the task log file with the attrep_cdc_changes table to verify that all existing changes have been captured.
10. Stop the task.
11. Select Commit Timestamp (CTS) from the Trigger-based mode drop-down list.

12. Open the Advanced Run Options dialog:

    1. Select Tables are already loaded. Start processing changes from.

    2. In the Source change position field, enter V3:0:0:0:0:0 and then click OK.

    Changes will now be captured from the new attrep_cdc_changes_cts table.

Creating the new artifacts manually and switching to Commit Timestamp (CTS) mode

1. Stop the task using the SAP HANA endpoint in Log Table mode.
2. Select Commit Timestamp (CTS) from the Trigger-based mode drop-down list.

3. In the Advanced tab of the endpoint settings, clear the Create CDC artifacts check box.

4. Open the Advanced Run Options dialog:
   1. Select Metadata only and then select Create missing tables and then stop.
   2. Click OK and wait for the task to complete.
5. Browse to <replicate_installation>\data\tasks\<task_name>\scripts. The following files should now be in the scripts folder:
   • Script for creating the CDC changes table: TABLE_attrep_cdc_changes_cts.sql
   • Scripts for creating the triggers: attrep_cdc_v3_<source-table-name+id>
6. Run all the scripts in SAP HANA studio to create the relevant artifacts in the specified artifacts' schema.
   
7. Verify that the following artifacts have been created in the SAP HANA database:
   • A new table called attrep_cdc_changes_cts.
   • All of the source tables have new triggers with "V3" in their name 
8. Delete the old triggers. Old triggers will not have '"V3" in their names.
9. In the Advanced tab of the endpoint settings, Select Log Table from the Trigger-based mode drop-down list.
10. Resume the task and wait for it to complete capturing changes. You can compare the stream position in the task log file with the attrep_cdc_changes table to verify that all existing changes have been captured.
11. Once all changes have been captured, stop the task.
12. Select Commit Timestamp (CTS) from the Trigger-based mode drop-down list.

13. Open the Advanced Run Options dialog:

    1. Select Tables are already loaded. Start processing changes from.

    2. In the Source change position field, enter V3:0:0:0:0:0 and then click OK.

    Changes will now be captured from the new attrep_cdc_changes_cts table.

Log-based CDC

Log-based CDC is faster than Trigger-based CDC and has a smaller system footprint. When Log-based CDC is selected, Replicate will read the changes directly from the SAP HANA logs. In order for Replicate to be able to do this, you need to specify the location of the logs in the Redo Log Shared Folder and Backup Log Shared Folder fields respectively (both fields are mandatory).

If Replicate Server is not installed on the SAP HANA machine, the redo logs and backup logs must reside in Shared Folders on the SAP HANA machine.

Log Access Preferences

You can determine which logs Replicate will read the changes from. If the logs are encrypted, you also need to provide the encryption settings as described in Encryption Settings below.

From the Read changes from drop-down list, select one of the following:

• Prioritize backup log - This is the default mode as it provides best performance while also offering greater efficiency. In this mode, Replicate will first try to capture the changes from the backup log. If the changes do not exist in the backup log, it will try to capture them from the online redo log instead.

  When this option is selected, you also need to specify the locations of the Redo log shared folder and the Backup log shared folder.

• Backup log only - In this mode, Replicate requires access to the backup log only. Changes will be captured from the backup log as soon as they become available. The backup interval and the size of the backup files will impact the capture time.

  When this option is selected, you also need to specify the location of the Backup log shared folder.

• Redo log only - This mode should not be used in production environments. In this mode, Replicate requires access to the redo log only. Changes will only be captured if they exist in the redo log. In the event that the changes are not available in the redo log, the task will need to be reloaded to ensure that all data and changes from the source are available on the target.

  When this option is selected, you also need to specify the location of the Redo log shared folder.

Encryption Settings

Replicate can decrypt encrypted redo logs and backup logs. The encryption root keys can either be retrieved automatically by Replicate or provided by the user in the designated fields.

• Automatically retrieving the encryption root keys - This method is preferable when the encryption root keys frequently change or if the encryption root key backup files are not accessible from the Replicate machine. This method also requires the Replicate user to be granted the ENCRYPTION_ROOT_KEY_ADMIN permission (for retrieving the encryption root keys).
• Manually providing the encryption root keys - This method is preferable when the encryption root keys rarely change and when the encryption root key backup files are accessible from the Replicate machine. This method may also be preferable for organizations that do not want to grant the ENCRYPTION_ROOT_KEY_ADMIN permission to the Replicate user.

The following settings are available:

• Encrypted redo log - Select this option if the redo logs are encrypted. If you need to manually provide the encryption root keys, specify the path of the redo log encryption root key backup file in the Log root key backup (.rkb) file field. Otherwise, leave this field empty.
• Encrypted backup log - Select this option if the backup logs are encrypted. If you need to manually provide the encryption root keys, specify the path of the backup log encryption root key backup file in the Backup root key backup (.rkb) file field. Otherwise, leave this field empty.
• Root key backup password - The password for reading the root key backup files.

Internal parameters

Internal parameters are parameters that are not exposed in the UI. You should only use them if instructed by Qlik Support.

To add internal Qlik Replicate parameters:

1. Click the Internal Parameters link.

   The Internal Parameters dialog box opens.

2. In the edit box, type the name of the parameter you need to add and then click it.
3. The parameter is added to the table below the search box with its default value.
4. Change the default value as required.
5. To reset the parameter value to its default, click the "Restore default value" icon at the end of the row.

More options

These options are not exposed in the UI as they are only relevant to specific versions or environments. Consequently, do not set these options unless explicitly instructed to do so by Qlik Support or product documentation.

To set an option, simply copy the option into the Add feature name field and click Add. Then set the value or enable the option according to the instructions you received.

Settings summary

You can view a summary of your settings by clicking the Setting Summary link. This is useful if you need to send a summary of your settings to Qlik Support.