Step 3: Create a replication task to transfer the staged data to the target
After the data is updated in the Log Stream Staging folder, you then need to create a Replication task to transfer the changes from the Log Stream Staging folder to the final target endpoint.
If you stop the task and define a metadata transformation for one of the tables (such as dropping a column), make sure the DROP and CREATE table option is selected (the default) in the Task Settings' Full Load Settings tab before resuming the task.
To do this, you must first duplicate of the source endpoint that has the data you want to replicate. Then, you can create a Replication task in which the source endpoint is the duplicated source endpoint of the Log Stream Staging task and the target is any required target endpoint.
Duplicate the source endpoint and add it to the log stream staging task
-
When duplicating a source endpoint for use in a Replication task, you can define credentials that are different to those used in the Log Stream Staging task.
-
Instead of duplicating the source endpoint, you can create a new endpoint and give it the same properties as the existing one.
To duplicate a specific endpoint and connect it to the Log Stream Staging task:
- In the left panel of the Manage Endpoint Connections window, select the original source endpoint defined for the Log Stream Staging task.
- Click the Duplicate button.
- In the General tab, do the following:
- Replace the name of the source endpoint prefixed by “Copy of …” with a meaningful name for the duplicated endpoint (for example, Oracle1 with Log Stream).
Select the check box Read changes from log stream staging folder.
Information noteThis check box only appears if the duplicated source is connected to the Log Stream Staging endpoint.
From the Log stream staging task drop-down list, select the name of the relevant Log Stream Staging task.
Information noteAll Log Stream Staging tasks that are related to the duplicated source are provided in the drop-down list. If a task was never run, "Not Run" is displayed next to its name.
- Click Test Connection to determine if you are connected to the Log Stream staging task and if the information you entered is correct.
- Click Save and then Close.
Create the replication task using the duplicated source endpoint
Before you begin: Make sure that the Log Streaming Staging task is running and that it includes all the tables you need for the Replication task.
To create a new Replication task using the duplicated source endpoint
- In the Tasks view, click New Task. The New Task dialog box opens.
- Specify a meaningful name for the replication task.
- Choose the Unidirectional replication profile option.
- If required, select the Full Load or Store Changes task options.
- Click Apply Changes, and then OK.
-
Select the required source with Log Stream endpoint from the list of endpoint connections, and drag it to the top circle in the endpoints map.
Example:
Information note-
A Log Stream indication is added to the source endpoint icon.
-
The Task Name links directly to the Log Stream Staging task.
-
- Select the required target endpoint from the list of endpoint connections, and drag it to the bottom circle in the task map.
- From the right pane in the Console, click Table Selection and select the tables that you want to replicate. You can select any of the tables that were included in the Log Stream Staging task.
-
Click the Run button to execute the Replication task.
The Replication task will load the tables to the final target endpoint, replicating the full load from the original source endpoint and the changes from the Log Stream Staging folder.
Replication task limitations and considerations
- The Bidirectional replication profile is not supported.
- In the Target Metadata tab of the Task Settings dialog box, if you want to enable the Replicate LOB columns option, the Limit LOB size to value must be the same as the value specified in the Log Stream Staging task. Note however that if the Replicate LOB columns option is disabled for the Log Stream Staging task, it cannot be enabled in the Replication task.
- In the Table Settings window, most tabs are available and functional with the exception of the LOB Column Handling tab which is not available (at table level).
- The source_lookup Data Enrichment function is not supported. For more information on this function, see Data Enrichment functions
- The Source change position (e.g. SCN or LSN) Advanced Run option is only supported with the Oracle source endpoint.
- When the Use all table partitions Parallel Load method is enabled, source data in a partition that was created after the Log Stream Staging task started, will not be replicated to the target defined for the Replication task.