CDP overview

Qlik Application Automation for OEM has a unique feature to improve the implementation of data synchronization templates called the CDP (Central Data Platform).

The CDP should not be confused with a Customer Data Platform.

The Qlik Application Automation for OEM CDP is a central database used as intermediate storage (cache) to implement data synchronization templates.

Benefits of the CDP

• Unified data model
• Ability to re-use automations in multiple Bundle templates (thanks to the unified data model)
• Central storage of the id's of an object from both source and destination (the CDP object has a direct link to its source object and its destination object)
• Smart detection of changes which eliminates endless loops in 2-way sync: if you update a record in the CDP and it has no changes, the timestamp last update will not increase and no webhook will be triggered.
• CDP Viewer (under menu item My CDP) to see and search all the data stored in the CDP

CDP Datamodel

The CDP has a fixed data model. The goal is to unify integrations by mapping data from a given cloud platform to the CDP unified data model.

Custom fields can be added to objects, in order to store data that does not fit the CDP unified data model.

The CDP also has a table for custom objects, this table can be used to store any type of data, e.g. invoices, orders etc. A custom object has a key which should be unique and a value. The value can contain any object.

CDP tables

New tables are added on a regular basis. Check the CDP connector in the automation editor for up-to-date information on the datamodel. Below is a list of current tables (object types) and for each table the FK's (foreign keys, links to other tables).

CRM tables:

• Companies
• Contacts
  • optional FK's: company_id

EventTech tables:

• Events
• Attendees
  • optional FK's: event_id
  • extends Contact
• Exhibitors
  • optional FK's: event_id
  • extends Company
• Sessions
  • optional FK's: event_id
• Session registrations
  • required FK's: session_id, attendee_id
• Speakers
  • optional FK's: event_id
  • extends Contact
• Sponsors
  • optional FK's: event_id
  • extends Company
• Exhibitors
  • optional FK's: event_id
  • extends Company

Cloud Telephony tables:

• Calls

General/other tables:

• Invoices
• Custom objects

How the CDP is used

A typical integration template using the CDP will consist of at least 2 automations:

• One automation to read data from a source platform and store it in the CDP
• One automation to read data from the CDP and send it to a destination platform

The automations are combined in an integration template using the Bundle Templates concept. A Bundle is a grouping of automations that form one template. See the article on Bundles for more information.

Thanks to the CDP, automations can be reused in multiple Bundles. Let's say you have to create 10 integrations from various CRM's to a marketing automation tool called ACME. Without the CDP, you would build 10 integration template automations:

• Salesforce –> ACME
• Hubspot –> ACME
• MS Dynamics –> ACME
• Etc.

With the CDP, you would build 10 Bundles. Example:

• Bundle Salesforce:
  • automation Salesforce –> CDP
  • automation CDP –> ACME
• Bundle Hubspot:
  • automation Hubspot –> CDP
  • automation CDP –> ACME
• Bundle MS Dynamics:
  • automation Dynamics –> CDP
  • automation CDP –> ACME
• Etc.

As you can see above the automation CDP –> ACME can be reused in each template.

CDP Basic usage

This section covers the various uses of the CDP.

Reading data

We'll use Contacts as example, but the available Blocks are identical for other object types:

• List contact: returns all contacts
• Search contacts: search on one field
• Search contacts advanced: ability to perform AND/OR on multiple fields
• Get contact: get one record by id
• Get contact by field: get one by any field (e.g. email = john@acme.com)

Webhooks

You can build Triggered automations that listen to CDP Webhooks. The CDP will trigger a Webhook when data is added, updated or deleted. Example:

Use Exclude from source to ignore updates coming from one source.

The Test payload is the incoming payload that will be used when you test the Triggered automation with the Test Webhook button. The default test payload shows you which data will be available. Note that the payload will include e.g. the source of the event (e.g. Salesforce).

The CDP will not trigger a webhook when a record is updated but no fields were changed.

Writing data

The CDP uses Upsert blocks that will perform an insert or update. The Where section in the inputs are used to define how to find existing records to update. In the following example, the contact with email john@acme.com will be updated with a new email address john.doe@acme.com:

External id's and sources

The CDP can store the id of the record from an external platform, this is called the external_id in the CDP, and is always combined with a source. The source can be anything, but make sure to use the identical naming throughout.

Each record can have multiple external_ids, one for each source. The external_ids are stored in the object as follows (even if there is only one):

{
  "id": "...",
  "email": "john@doe.com",
  "external_ids": {
        "Hubspot": "12345",
        "Salesforce": "6789"
    }
}

As you can see, each source becomes a key under the property external_ids. The external_id can be retrieved with following path (assuming the source is Salesforce):

{ $.getContact.external_ids.Salesforce }

Records can be upserted using their external_id as follows:

Note that the external_id is mapped both in the input field (External Id) and in the Where section. The same is true for the source:

Custom objects

The Table Custom objects in the CDP can be used to store any type of object, e.g. an Order or Product. Each custom object has a key which is unique and a value. The value can be any object with properties/values. Example to insert/update a custom object using its key:

You can also use the more generic Upsert Custom object, which has a Where section to define the fields to search for existing records.

Similar to other objects in the CDP, can Custom objects also have external_ids (always combined with a source).

CDP scope

The CDP is private per customer account, and by default the data is shared between all automations from that account that read data from the CDP and write data to the CDP.

You can also limit the scope by adding a scope to all CDP blocks that you use (including CDP Webhooks blocks). A scope can be anything (e.g. the name of a project, or a guid). A very common scope is a bundle guid, see below.

Limiting the scope to one bundle

Use the placeholder { bundleguid } as scope to limit the scope of all automations in a Bundle to one single bundle. This is needed to avoid processing data from other bundles, in case more than one integration is active in a single account.

An additional benefit is that when an integration is deleted and activated again, this will be a new bundle and thus also a new scope. This avoids CDP data from a previous installed integration being used in the new instance.

Note: a template should delete all its data from the CDP when the instance is deleted by the user. This is accomplished by using the Delete flow in the template, combined with blocks Delete records by scope.