Replication Overview

Skyvia replication operation is used to create a copy of cloud application data in a relational database and keep it up-to-date. When performing replication, you define application objects and their fields to copy and configure replication options. Unlike Import, replication does not support loading data of different structure in source and in target and using any custom mapping; however, it is much simpler to configure.

Replication can be used when you need to backup cloud application data to a relational database, archiving or historical data that is no more a subject to change and is not needed in the cloud application. It can be useful in different data integration scenarios, or just for performing data analysis and reporting with powerful data analysis and reporting tools, developed for relational databases.

Skyvia can create corresponding database tables automatically with optional foreign keys, and, if necessary, drop existing tables before creating. It allows you to filter data to replicate.

note Note

Skyvia doesn't create foreign keys, corresponding to many-to-many CRM object relations, in the database.

For more information see the replication tutorial - Setting Up Replication of Salesforce Accounts and Contacts to SQL Azure.

 

ExpandedToggleIcon        Incremental Updates

If you need to keep the copy in the database up-to-date, you can schedule package for automatic execution and use the Incremental Updates feature, which allows copying only the data that was changed in the cloud CRM since the last package execution, and then applying only the necessary changes to the database. In case the Incremental Updates check box is selected, table creation and dropping is performed only for the first package run if the corresponding check boxes are selected. See Replication Package, the Replication Package Settings in Details section, for more information.

Note that in some sources Incremental updates are not supported for some of the objects. For incremental updates, an object must have fields, storing its  creation and modification time. See details for each source in the Connections section. Both fields with the creation and modification time must be present. (The only exception is G Suite, for it one Updated field is enough).

If you select objects without such fields in a replication package, the result depends on whether you replicate data to a database (like SQL Server, SQL Azure, MySQL, etc.) or a cloud data warehouse - Amazon Redshift, Azure SQL Data Warehouse, or Google BigQuery. A package, replicating data to a database, will not run. When running such a package, you will get an error that the source object does not have the corresponding column.

A package, replicating data to a data warehouse will run successfully and load data. However:

If an object does not have both of the fields, Skyvia will display a warning for such object in a replication package editor. Only the first replication run will load data from this object to a data warehouse. Subsequent replication runs won't load any data from this object.
If an object does not have a field, storing record creation time, but has a field, storing record modification time, no warnings are displayed. Subsequent replications will update the cloud data warehouse records, modified in the cloud app, but won't add new records, created in the cloud app.
If an object does not have a field, storing record modification time, but has a field, storing record creation time, no warnings are displayed. Subsequent replications will load new records, created in the cloud app, but won't apply any modifications to existing records for updates, made in the cloud app.

Delete Operation

For some data sources Skyvia deletes a record from the database when a record is deleted in the cloud source when incremental updates are enabled. This feature is supported only for data sources and objects, for which Skyvia can obtain the information about deleted records. For example, replication applies deletes for soft deleted records in Salesforce (but not hard deleted records), or for deleted records in Dynamics CRM if audit is enabled for them.