Click the Full Load Settings sub-tab to configure the following:
Full is ON/OFF.
Click this button to toggle full load on or off. The initial setting is determined when Adding tasks.
When full load is ON, Qlik Replicate loads the initial source data to the target endpoint.
Full load can be turned on or off at any stage even if change processing is on. Once the task begins to process changes, the full load on/off switch is used only as additional protection against accidental or unauthorized reload.
Target table preparation:
The option to drop or truncate the target tables is relevant only if such operations are supported by the source endpoint.
Replicate Control Tables will not be dropped. However, any suspended tables that are dropped will also be deleted from the attrep_suspended_tables Control Table if the associated task is reloaded.
- TRUNCATE before loading: Data is truncated without affecting the table metadata. Note that when this option is selected, enabling the Create primary key or unique index after full load completes option will have no effect.
ARCHIVE and CREATE table: A copy of the existing table will be saved to the same schema before the new table is created. The archived table name will be appended with a timestamp, indicating when the archiving operation occurred (e.g. Customers_20170605175601).Note:
Currently this option is only available for the Hadoop target endpoint.
- Do nothing: Existing data and metadata of the target table will not be affected. New data will be added to the table.
Replicate expects the source column data types to be compatible with the corresponding target column data types. If you choose either TRUNCATE before loading or Do nothing and one or more target data types are different than the data types for the corresponding source columns, use a transformation to convert the data types as required.
For information on creating data type transformations, see Defining transformations for a single table/view.
After Full Load completes, stop the task: You can set the task to stop automatically after Full Load completes. This is useful if you need to perform DBA operations on the target tables before the task’s Apply Changes (i.e. CDC) phase begins.
During Full Load, any DML operations executed on the source tables are cached. When Full Load completes, the cached changes are automatically applied to the target tables (as long as the Before/After cached changes have been applied option(s) described below are disabled).
This feature is not available for bidirectional replication tasks.
Select Before cached changes have been applied to stop the task before the cached changes are applied and/or After cached changes have been applied to stop the task after the cached changes are applied.
Selecting the Before cached changes have been applied option will stop the task immediately after Full Load completes. Selecting the After cached changes have been applied option will stop the task as soon as data is consistent across all tables in the task.
When configuring Replicate to stop the task after Full Load completes, note the following:
The After cached changes have been applied option is not supported with all file-based and Hadoop-based target endpoints, namely:
- File-based: File, Amazon S3, Microsoft Azure ADLS, and Google Storage.
- Hadoop-based: Hadoop, Hortonworks Data Platform, Amazon EMR, Microsoft Azure HDInsight , Google Dataproc, Cloudera Data Platform (CDP) Private Cloud, and Microsoft Azure Databricks.
- The task will stop after Full Load completes even if there are no cached changes to apply.
- Choosing to stop the task before cached changes have been applied may adversely affect performance, since the cached changes will only be applied to tables (even those that have already completed Full Load) after the last table completes Full Load.
- When the Before/After cached changes have been applied option is selected and a DDL is executed on one of the source tables during the Full Load process (in a Full Load and Apply Changes task), Replicate will reload the table. This effectively means that any DML operations executed on the source tables will be replicated to the target before the task stops.
When working with the File Channel endpoint, these options should be set in the remote File Channel task and not in the local File Channel task.
For more information on the File Channel endpoint, see Using the Qlik Replicate file channel.