Synchronization Package

Synchronization package is a specific kind of a package, which performs bi-directional data synchronization between cloud applications and relational databases.

To create a Synchronization package, navigate to the Integration Packages page by clicking Integration in the menu on the left side of the page, click the New button and then click Synchronization. This will open a Synchronization Package editor page.

In the import package editor, you will need to perform the following actions:

1.Specify the data sources you import data from and to and connections to them.
2.Create synchronization tasks for the package.
3.Optionally schedule the package for automatic execution.

 

ExpandedToggleIcon        Synchronization Requirements and Supported Sources

Skyvia supports synchronization of cloud applications and relational databases. It does not support synchronization of cloud data warehouse services, like Amazon Redshift, Google BigQuery, or Azure SQL Data Warehouse.

For synchronizing database data, the primary key columns of the database tables must be autogenerated. For example, you may use autoicrement/identity columns for a primary key or create triggers to generate and assign primary key values.

Synchronized cloud objects must have fields, storing information when an object was created and last modified. Both of the fields must be present in an object. In some cloud applications some of the objects may have only one of such fields or don't have any. Such objects cannot be added to a synchronization package. If you try to select such an object for synchronization, the corresponding error message is displayed.

Note that some cloud applications, like Podio or SendPulse, don't have objects with both such fields at all. These data sources cannot be synchronized at all.

 

ExpandedToggleIcon        Package Connections

When creating a synchronization package, first you need to select source and target connections.

Synchronization is bi-directional, so the only difference between the Source and Target is the following: in case of conflicts, source data changes have priority over the target data changes. If the same records was modified in both source and target between synchronizations, the source values are applied to the corresponding records in the target.

You can specify package connections in the following way:

1.In the Source header click Select and specify the type of the source data storage. Then repeat the same with the Target header.
 
target-type-selection-sync
 
 
2.Select the Source Connection for the package. If you haven't created the source connection yet, click the New button and create a new connection (learn how).
 
select-source-salesforce-connection
3.Select the Target Connection
 
select-target-connection-sync-sql

 

ExpandedToggleIcon        Creating Tasks

To create an synchronization task, click the + Add task button and configure a new synchronization task in the Synchronization Wizard as described in the How to Create Synchronization Task topic.

 

ExpandedToggleIcon        Package Settings

You can rename your package by clicking the btn-edit-name Edit name button near the automatically generated package name. By default, the package name is generated based on the synchronization task operations and target object or table names.

You can schedule your package for automatic execution. See Scheduling Packages for more details.

Dedicated Account

If the source or target are Salesforce, Dynamics CRM, or SugarCRM, and synchronization uses a CRM account that is not used for any other data update operations, select the Dedicated [CRM] account for synchronization check box (where [CRM] can be Salesforce, Dynamics CRM, or SugarCRM). This check box means that this account is used only for the synchronization package, and changes, that are made by this account, are ignored by the synchronization.
 
If this check box is selected:
 
a) The first synchronization synchronizes all the data.
b) If users, other than the user specified in the synchronization package connection, make changes to application data during the synchronization operation or after it, these changes are synchronized in the next synchronization operation.
c) Any changes, performed by the user specified in the synchronization package connection are ignored. Skyvia assumes that this application user account is dedicated for the synchronization operation and does not perform any other data changes.
 
If this check box is not selected:
 
a) The first synchronization synchronizes all the data.
b) Any changes to CRM data performed during the synchronization operation are ignored. If other side is a relational database, all the changes to it performed during the synchronization operation are also ignored. It's highly recommended not to edit the data being synchronized during the synchronization operation.
c) Any changes, performed by the user specified in the synchronization package connection or any other user between the synchronization operations, are synchronized.
 
We recommend creating a dedicated cloud application account for the synchronization operation and selecting the Dedicated [CRM] account for synchronization check box.
 
 
dedicated-account

After you have configured your package, click the Save button.

 

ExpandedToggleIcon        Synchronization Package Parameters

In Skyvia, Synchronization packages have the following two parameters.

 

Parameter

 

Description

LastSyncTime

The time since the last synchronization. When synchronization is executed, Skyvia sets this parameter to the current time, and the next package execution  will synchronize the changes made since the time, specified in this parameter. You can reset this parameter in order to reload all the data.

InitTrackingObjects

A Boolean value, determining whether to create tracking tables and triggers in the synchronized database. It is true before the first package run, when they are not created yet. Package execution sets it to false. You can reset it when you need to re-create the tracking tables and triggers in the database.

sync-parameters

 

ExpandedToggleIcon        Editing Existing Package

Editing existing package is performed via the same package editor page with the same interface elements as when creating a new package. To edit a package, on its Details page click Edit.

In the package editor, you can change connections, add, edit, or delete tasks, enable, configure, or disable package schedule, etc.

Editing or Deleting Tasks

When editing an existing synchronization package, which have already run, you can freely delete tasks and unmap fields. However, when mapping new fields or changing mapping for already mapped fields, you should note that next synchronization run won't resync all the records with the new mapping automatically. Next synchronization run will only sync records that were modified since the previous package run, so the new mapping will be applied only for these records.

If you want to resync all the records with the updated mapping, you can choose one of the following workarounds:

If your data are synced - you have the same records on both sides, clear the LastSyncTime parameter and delete records on one of the sides. Then the synchronization will be performed as in the first time, and it will load all the records from another side to the side with the deleted records.
 
Note: DON'T forget to clear the LastSyncTime parameter or you may lose your data.
Perform a mass update of your records in one of the sources with the same values that are already in this data source. For example, you may use our Query tool and perform an UPDATE statement. Here is an example of such statement for Salesforce Pricebookentries:
 
UPDATE PricebookEntry SET IsActive = IsActive
 
This statement does not actually modify data, because it assigns an already assigned value for records, but the records will be considered modified, and the Synchronization package will resync them to the other source.

When using any of these workaround, please be careful, and act only if you know what you are doing. Incorrect actions may cause data loss.

Adding Tasks

Adding a task to an existing package that already has run is a more complex case. If one or both of the sources are databases, you will definitely need to reset both of the synchronization package parameters and run the package as in the first time. If you have synchronized data, you may need to truncate previously synchronized tables on one of the sides to avoid creating duplicates.

If both of the synchronized sources are cloud applications, then using the second workaround for the added object is possible.

Anyway, we recommend to plan your synchronization packages in advance and add all the necessary tasks when creating them. In case you need to add tables/objects to synchronization, it may be better to create a completely new package.