Connectors


Connectors are integrations between Hull and external data sources such as:

Connector Limits

The number of Connectors in your Hull Organization is limited by your billing plan. Each connector instance is counted as 1, even if you have multiple connectors of the same type.

You can view the full list of Hull Connectors here. Hull Connectors are also Open Source here and here, and available to be self-hosted.

How Connectors Work

Connectors sync Users, Accounts, Events, Attributes, and Segment data into Hull, out of Hull, or both. To sync data into Hull, Connectors send data to the Firehose API to ingest. To sync data out of Hull, Connectors subscribe to updates published by the Notification API.

Note: Connectors are not the only method of syncing data in or out of Hull. Other methods include using platforms and our import and export APIs

Using Connectors in Hull

Connectors and their status can be viewed through the Connectors page in the Hull dashboard. There, you can install new Connectors through the Connector Gallery.

Each Connector has an individual Overview page. Here you can view:

  • Overview - See API calls made by the connector, Outgoing Backlog (read more here), revisions of connector settings (read below), notifications of errors, warnings and hints related to connector dataflow and settings
  • Settings - Configure data flows by mapping Segments, Attributes & Events, managing identifiers, and name/describe each Connectors behavior.
  • Logs - View and query logs for a Connector
  • Workspace - custom configuration for a Connector, accessible via the blue button on the left sidebar, it’s custom for each connector, but here are the most common examples
    • Credentials & Actions - manage authentication and connection with each service
    • Code Editor - view, write, edit & delete code to operate on the Connector

The exact setup varies between types of Connectors.

Connector Overview

Pausing Connectors

At any time after installation, connectors can be paused. This will stop processing any user updates, user events or account updates. All of those updates are accumulated in the Outgoing Backlog for this connector, and wait until the connector is resumed.

IMPORTANT: Updates in the Outgoing Backlog expire after a few weeks’ time, so if you keep your connector paused for a longer period of time, some older updates may be lost.

When a connector is paused it will also DROP any incoming records which won’t be stored nor processed by our platform. The incoming traffic will be only ingested again when the connector is resumed. Most of synchronization connectors allow you to perform full fetch of objects from the 3rd party service which depending on the connector can backfill information about Users and Account but some of User Events may not be replayable that way.

IMPORTANT: there are certain incoming records which may not be recoverable even if “fetch all” operation is available on certain connectors

When a connector is paused, it still can process manual batches of users or accounts sent to it. Connector settings can be changed as usual, and if available, the connector can be re-authorized against the 3rd party service.

When / Why pause a connector?

Whenever you see invalid data being send to or from the 3rd party service. When a connector is paused you can safely adjust it’s settings, push sample users to confirm the correctness of data and enable the connector again when ready.

Connector revisions & commit message

Since Connector settings are crucial for your dataflow configuration, each change can have an important impact. For this reason, each change of the connector settings is stored as a new revision. Those revisions are visible on the overview of the connector. You can easily see the number of the revision, name of the person who did the change, the time of the change, and if you click it you will see which settings changes, the previous and new value.

Connector Changelog

Additionally, whenever an operator saves connector settings, he will be prompted to add a commit message and description which allows adding more context to the change.

Limitations

Revisions are currently subject to few limitations:

Detailed diff shows internal values

The diff window which you can see in the sliding sidebar shows the internal value of the settings. And in some cases, they are not consistent with the labels which are visible in the settings UI. Segments filter, for instance, will show only segment ids and not names. Similarly attributes mapper will also show internal names of attributes and fields. In case of difficulties in reading those values please reach out to us at support@hull.io

Authorizing a connector does not result in visible revision

When a connector is authorized via oAuth flow a new revision is created in the backend but it is still not visible on the dashboard, so there is no way to tell who did the authorization.

Some revision numbers are skipped

Some connectors store extra metadata behind the scenes. This information is internal to the connector and is used to optimize the synchronization with the 3rd party system. That data is also versioned, but not visible in the dashboard. This is the reason why the revision numbers between changes introduced in the settings may not be consecutive.