Replication frameworks
Frameworks are predefined and automated processes that enable to perform a standard replication or a transformation to the data before their replication.
Information noteNote: The Transformation and selection tool is not needed at
this step. However, if necessary, after the transformation has been configured, it is still
possible to edit the script with the help of this tool.
Frameworks are selected when adding new source tables to a model.
If you want to process:
- standard replication without any transformation, use the Free framework.
- replications that involve a transformation, use the other frameworks: Staging framework, Audit framework, Aggregate framework, BigData framework, Date I/O framework.
As frameworks are associated to each table, you can use various frameworks in the same model (or source database).
Using various frameworks for the same source database
Staging and BigData frameworks can be used in parallel to replicate the same database. In this case, the target database will include the mirrored database plus the history of all data changes.Information noteWarning: It is not possible to use Free and Big Data frameworks in
parallel.
In case you have big storage capacities and you need to produce predictive modeling,
the Big Data framework can be used in parallel with the Audit one.
Information noteWarning: In
this case, as frameworks are used for separate reasons, it is necessary to replicate the
source table on two separate target tables. On the contrary, if you use Snowflake as a
streaming pipeline only, data sent in a JSON file must be replicated on a single
table.