Connectors


Connectors are integrations between Hull and external data sources such as:

Connector Limits

The number of Connectors in your Hull Organization is limited by your billing plan. Each connector instance is counted as 1, even if you have multiple connectors of the same type.

You can view the full list of Hull Connectors here. Hull Connectors are also Open Source here and here, and available to be self-hosted.

How Connectors Work

Connectors sync Users, Accounts, Events, Attributes, and Segment data into Hull, out of Hull, or both. To sync data into Hull, Connectors send data to the Firehose API to ingest. To sync data out of Hull, Connectors subscribe to updates published by the Notification API.

Note: Connectors are not the only method of syncing data in or out of Hull. Other methods include using platforms and our import and export APIs

Using Connectors in Hull

Connectors and their status can be viewed through theย Connectorsย page in the Hull dashboard. There, you can install new Connectors through the Connector Gallery.

Each Connector has an individual Overview page. Here you can view:

  • Overview - See API calls made by the connector, Outgoing Backlog (read more here), revisions of connector settings (read below), notifications of errors, warnings and hints related to connector dataflow and settings
  • Settings - Configure data flows by mapping Segments, Attributes & Events, managing identifiers, and name/describe each Connectors behavior.
  • Logs - View and query logs for a Connector
  • Workspace - custom configuration for a Connector, accessible via the blue button on the left sidebar, it’s custom for each connector, but here are the most common examples
    • Credentials & Actions - manage authentication and connection with each service
    • Code Editor - view, write, edit & delete code to operate on the Connector

The exact setup varies between types of Connectors.

Connector Overview

Disabling Connectors

โ€‹ At any time after installation, connectors can be disabled. This will stop processing and DROP any user updates, user events or account updates. โ€‹ When a connector is disabled, it still can process manual replays of users or accounts sent to it. Connector settings can be changed as usual, and if available, the connector can be re-authorized against the 3rd party service. โ€‹

IMPORTANT: User Events cannot be manually replayed like attribute updates. If your connector relies on the events, the data will be lost when the connector is disabled. โ€‹

When a connector is disabled it will also DROP any incoming records which won’t be stored nor processed by our platform. The incoming traffic will be only ingested again when the connector is enabled. Most of the synchronization connectors allow you to perform full fetch of objects from the 3rd party service which, depending on the connector, can backfill information about Users and Account, but some of the User Events may not be replayable that way at all. โ€‹

IMPORTANT: there are certain incoming records which may not be recoverable even if “fetch all” operation is available on some connectors โ€‹

When / Why pause a connector? โ€‹ Since the connector stays operational even when disabled, it can be helpful to disable it to finish or correct its setup. It may be important when observing invalid data being sent to, or from the 3rd party service. To prevent further data disruption, disabling a connector may be the safest way to correct the configuration. After the connector settings are adjusted, a sample of users can be pushed to confirm the quality of data. The connector can then be enabled again when ready.

IMPORTANT - CUSTOM CONNECTORS: the disable feature is available for all installed connectors, but if you are deploying your custom connectors you need to be aware that the feature may work differently for you and not all incoming traffic may be dropped. In case of questions reach out to connector developer and refer to connectors reference

Connector revisions & commit message

Since Connector settings are crucial for your dataflow configuration, each change can have an important impact. For this reason, each change of the connector settings is stored as a new revision. Those revisions are visible on the overview of the connector. You can easily see the number of the revision, name of the person who did the change, the time of the change, and if you click it you will see which settings changes, the previous and new value.

Connector Changelog

Additionally, whenever an operator saves connector settings, he will be prompted to add a commit message and description which allows adding more context to the change.

Limitations

Revisions are currently subject to few limitations:

Detailed diff shows internal values

The diff window which you can see in the sliding sidebar shows the internal value of the settings. And in some cases, they are not consistent with the labels which are visible in the settings UI. Segments filter, for instance, will show only segment ids and not names. Similarly attributes mapper will also show internal names of attributes and fields. In case of difficulties in reading those values please reach out to us at support@hull.io

Authorizing a connector does not result in visible revision

When a connector is authorized via oAuth flow a new revision is created in the backend but it is still not visible on the dashboard, so there is no way to tell who did the authorization.

Some revision numbers are skipped

Some connectors store extra metadata behind the scenes. This information is internal to the connector and is used to optimize the synchronization with the 3rd party system. That data is also versioned, but not visible in the dashboard. This is the reason why the revision numbers between changes introduced in the settings may not be consecutive.