Map lowercase tagged sources to capitalized form during ingestion - #4417

Changes

This work is in preparation of a bigger project - Acquisition Channel.

This PR changes how we handle manually tagged acquisition data. When links are manually tagged, usually the utm_source parameter has a value like google or facebook. However, when we automatically detect sources based on the Referer header, we store the source field as Google or Facebook. So then we have dashboards that have both sources google and Google where the first is manually tagged traffic and the second is automatically detected.

However, both are coming from the same source so we would like to merge the lowercase and the capitalized entry in sources.

The way I've decided to tackle it here is to build a lookup table from the RefInspector library that stores all sources in a table like:

lowercase(source) -> source

We use this table to lookup google -> Google during ingestion, for example.

The logic is that if the manually passed utm_source matches the lowercase form of any known traffic source, it gets transformed and stored as the capitalized form like automatically added sources.

We have also discussed a data migration for historical data to apply the same transformation. I'll look into that next.

Tests

• Automated tests have been added

Changelog

• Entry has been added to changelog

Documentation

• Docs have been updated

Dark mode

• This PR does not change the UI