Data UnificationUpdated 03/01/2019


The goal of customer data integration is to ensure every tool owned by every team has a complete, up-to-date profile of each lead & customer.

As teams grow and introduce more tools into the system, it becomes exponentially complicated to achieve this goal using the same data integration methods.

  • Manual integration: Copy-and-paste or exporting & re-uploading CSVs is time-intensive and often outdated at scale. It is complex between multiple different tools
  • Automated integration: Integrations don't all exist, or sync the exact data that's needed, or sync in both directions.
  • Engineering integration: Technical resources aren't always available to build & maintain integrations between (often changing) APIs.

There's also the challenge of complexity with adding additional tools to a system. To make sure every tool is kept up-to-date, every tool must be connected to every other tool.

  • 3 tools needs 3 integrations
  • 5 tools needs 10 integrations
  • 10 tools needs 45 integrations

(The maths for this; if you have n tools then you need (n*(n-1))/2 integrations)

This becomes very complicated and opaque at scale.

pairwise-customer-data-integration

Best practice: centralize your data integration

Instead of everything pointing to everthing else, centralize and unify your customer data through one single source of truth.

centralized-customer-data-integration

Universal tracking

Build a complete picture of all your leads & customers by tracking their entire customer journey and enriching profiles with all known data.

Unified customer profiles

Combine data from every source about each person and company into a single customer view. This requires an identity resolution strategy to match up data correctly.

Unified attributes

Instead of mapping synonymous attribute/property/trait between every tool and maintain many different (often conflicting) versions of the same data, compute a "unified" attribute to sync across all your tools.

Centralized data mapping

Control what data is fed into all your tools from one centralized data source - your single source of truth. Map data from your unified customer profiles (and unified attributes) to all your end tools.

Creating fallback strategies with Processor

To unify attributes, you need to compute a new attribute in Processor from all the possible sources of the same data type.

For example, you might have many competing possible sources of company name:

  • Hull Account Name (default auto-created Account attribute)
  • Salesforce Account (from sales rep input)
  • HubSpot Company (from form fill)
  • Clearbit Enrichment
  • Signup domain name

Your fallback strategy should lookup each of these sources in order (where they've been ordered by reliability and completeness), and populate the unified attribute with the highest order version with a value.

You unified attribute becomes the version of the attribute to map to external tools (instead of mapping Salesforce AND HubSpot AND Clearbit).

Here's an example of the fallback strategy for company name in Processor.

const userTraits = {};

// Company Name
if(_.get(user, "unified_data.company_name", null) === null {
    const companyNameFallbacks = [
      { dataObject: account, attribute: "name" },
      { dataObject: account, attribute: "salesforce.name" },
      { dataObject: account, attribute: "hubspot.name" },
      { dataObject: account, attribute: "clearbit.name" },
      { dataObject: account, attribute: "domain" }
    ];

  executeFallbackStrategy(userTraits, "unified_data/company_name", companyNameFallbacks);
}

traits(userTraits);

function executeFallbackStrategy(dataObject, attributeName, strategy) { 
  _.forEach(strategy, (s) => {
      if (!_.isNil(_.get(s.dataObject, s.attribute, null))) {
        _.set(dataObject, attributeName, _.get(s.dataObject, s.attribute));
        return false;
      }
    });
}