Today we're announcing a major improvement: We've leveled up our logging and the result is that we've considerably improved the clarity in how we represent the traffic flows into your Hull account.
Previously, each connector was responsible for logging the incoming data individually. This worked, but it lacked reliability and clarity in some specific but critical situations:
Today we're happy to announce that we're now generating reliable ingestion logging data from the platform's ingestion steps for all connectors and all data sources.
For those of you who are using custom forked connectors, this is big! You'll now start seeing ingestion logs for your connectors and you will be able to see what was sent and how it was ingested.
The SQL connector and others who are using our bulk import API weren't generating per-user and per-account logs, which made that data invisible in the logs. Now, they're generating the same data as the others.
The Hull platform has validation rules to ensure proper data is ingested. For instance, we filter domains at the account level so that we don't create accounts based off of "freemail" domains (think gmail.com). Sometimes, data is simply invalid and we're forced to drop the payload entirely. If such an error occurs now, you'll be able to see it and understand why.
We're now showing all the attributes that were received. This lets you search for them and see exactly what data was received and which user it was resolved to, making the process of analyzing data flow a much simpler process.
Within the logs, we're now showing the full list of attribute change made to a user. Not only do you see which data we captured, but you'll also see which attributes were modified, and the changes it made to the resolved profile. This makes it easy to identify loops and other data flow issues.