Transformations in Server-Side Google Tag Manager

In January 2020, when Google Tag Manager’s server-side tagging was first introduced to the general public at SuperWeek, I wrote a flurry of tweets, sharing my vision of a server-side tagging future.

In one of the tweets, I discussed how you could do these:

• Hit validation and fixing before the hit is sent to the endpoint
• PII and privacy controls for the requests before dispatch

Fast forward to today, over three years later, and we are finally treated to a feature that grants us scalable controls to properly interrupt data flows within server-side GTM.

I’m talking about TRANSFORMATIONS.

It’s a new feature and resource type in server-side tagging, which you can access through the new entry in the container resource navigator.

Transformations sit firmly between the client and tags, allowing you to include and exclude parameters from the event data object before tags can access it.

You can also use it to augment existing parameters. This is useful if you want to enrich the event data object with an appropriately designed Firestore pipeline, or something similar.

Read on for more information about how transformations work and how to use them.

Transformations in a nutshell

To understand transformations, you first need to recall how clients and tags interact in a server-side tagging container.

A client is a server-side tagging resource whose purpose is to digest an incoming HTTP request and ideally turn it into an event data object.

This event data object is then consumed by tags in order to perform their tasks, such as dispatching requests to third-party vendors. The event data object semantics are not standardized or enforced, but it is recommended that clients follow the schema outlined by Google.

For example, a client could take a URL parameter like &cid=12345.12345 and turn it into an event data object key named client_id with the value 12345.12345. Tags could then be coded or configured to take this client_id key from the event data object and populate their own outgoing requests with the value.

The best way to check what type of event data objects are generated by clients is to use Preview mode. When you select an event in the left-hand navigation, the Event Data tab will show what the client produced.

Why don’t you see events under all the requests? That’s because there either wasn’t a client to claim the request in question, or the client didn’t produce an event object using a specific template API.

The main problem with event data is that the client’s operations are fixed. The event data object it produces is the same for all tags that are configured to make use of it. If you want a tag to ignore certain fields in the event data object, or if you want a set of tags to mutate some of the values into another format, the tag template itself needs to support this, and not all tag templates do.

Furthermore, Google’s own clients, which the majority of users will default to as the main controllers of the incoming data streams, are black boxes and there’s not much you can do to control how they work.

For long, we’ve been waiting for a feature that would allow us to edit the event data object itself before tags get to consume it.

This is where transformations enter the stage.

With transformations, you can now create a rule that removes or modifies the values in the event data object before tags can access it.

Instead of having to make tag-specific exceptions, you can now modify the event data object itself. After doing so, any tag that is set to consume this transformation will be privy only to the modified event data object rather than the original one.

Importantly, transformations don’t permanently edit the event data object. They create a transformed clone of the object to be digested by whatever tags and whatever conditions you specify in the transformation (more about these below). Tags that are not affected by a transformation will have access to the original, unmodified event data object.

Create a new transformation

To create a new transformation, click the Transformations entry in the left-hand navigation of your server-side tagging container. You’ll see a list of your existing transformations. If you click the button labeled NEW, you can create a new transformation.

When you create a transformation, you’ll first need to choose its type.

Transformation types

Transformations come in three types:

• Allow parameters, which lets you allowlist the parameters that should end up in the event data object. This is very powerful, as parameters that are not allowed will be excluded from the event data.
• Exclude parameters, which lets you denylist the parameters that should not end up in the event data object. Parameters that are not excluded will be included in the event data.
• Augment event, which lets you add or modify event data parameters. This is where you would do data enrichment, sensitive data purges, and similar.

Allow parameters

When you choose Allow parameters as the transformation type, you’ll need to populate a table of rows, with each row corresponding to a parameter name in the event data object.

When this transformation is evaluated, the keys in the event data object are compared against the parameters listed in this transformation. If there’s a match, that key is kept in the event data object.

If there’s no match, then the key is dropped from the event data object for this transformation.

In the example above, the only allowed parameters are client_id, event_name, ga_session_id, and custom_timestamp. Other parameters are not included in this transformation.

This is a very powerful transformation type, as it lets you proactively exclude all parameters that you are not aware of or that you haven’t exclusively allowlisted for the tags that consume this transformation.

It’s also a risky transformation type because of this. Especially when working with third-party vendors, you might not know exactly which parameters the vendor needs due to poor documentation or lazy template design. The Allow parameters transformation is most potent when you’re working with a service for which you know exactly which parameters are required and which are optional.

Exclude parameters

Like Allow parameters above, parameter exclusion requires you to populate a table of rows, with each row corresponding to a parameter in the event data object.

However, this time when comparing keys in the event data object against this list, any key that matches between the two is dropped from the event data object. Keys that are not matched are kept.

In this case, client_hints and ip_override have been excluded from the final event data object in this transformation. That’s why they have empty values above whereas all the other parameter values are preserved.

While the allowlist is more proactive and pre-emptive as a measure, the Exclude parameters transformation lets you remove parameters that you know to be problematic.

Augment event

Event augmentation is what you’d use for data enrichment and for cleaning up the contents of the event data object. Instead of removing or keeping parameters, the augment event transformation type lets you add new keys to the event data object and/or modify existing values.

When you add parameter names and values into this transformation, any parameters that share the same name in the event data object get overwritten by the transformation’s value for the given parameter. If there is no pre-existing parameter with that name, a new one is added to the event data object.

In the example above, with the exception of currency, all the ecommerce parameters are retrieved from Firestore.

The cool thing is that the values you set support the full range of server-side Google Tag Manager’s variable capabilities, including things like asynchronous API calls and Firestore lookups.

While event augmentation is very useful for removing sensitive information from the page_location field, for example, its biggest potential lies in enriching the information in the event data object so that a multitude of tags can make use of the modification.

Transformation priority

It’s possible for multiple transformations to apply to a given event data key. For example, you might have a transformation in place that augments an event data key and also another transformation in place that excludes this key under certain conditions.

In this case it might be sensible to set the exclusion at a higher priority, so that the key isn’t unnecessarily augmented in cases where it’s already been excluded.

You can find the priority setting under the Advanced Settings of any transformation.

If you don’t specify a priority, then transformations are evaluated in this default priority order:

Allow parameters > Augment event > Exclude parameters

Matching conditions

Matching conditions allow you to enable the given transformation only when certain variable conditions are matched.

For example, given the Ecommerce augmentation example from above, it makes sense to only enable this transformation when the event_name value is purchase, as that’s the only event that would include the transaction ID required for fetching the data from Firestore (read more about this use case below).

If GA4 is your main incoming data stream, you could even add a Client Name condition to the transformation to make sure that other types of incoming streams don’t trigger the Firestore lookups.

If any of these matching conditions does not match, then the transformation is not evaluated and event data is not modified.

Affected tags

In addition to restricting transformations to work only under certain conditions, you can also apply them only to specific tags that you allowlist in the Affected tags settings.

In the example above, my Allow parameters transformation allowlists only a very restricted range of event data parameters for my custom analytics collector system.

Other tags can make use of the entire event data object, but this particular Custom Analytics Collector tag is only privy to the limited set allowlisted in this transformation.

Debug transformations

In Preview mode, the main way to see transformations at work is to select an event in Preview mode, and then open a tag that should be affected by a transformation.

Here, under Modified event data you’ll see what was available to the tag when it fired. If you check Show Original, you’ll see a diff of what the original event data object looked like.

The transformations that affect this tag are also listed. By clicking them open, you can see what the transformation is supposed to do, and you can also see another diff of “before” and “after” states of the event data object after the transformation does its job.

If there are multiple transformations affecting the current tag, then the “before” state might be impacted by prior (higher priority) transformations modifying it.

In the image above, this transformation only shows a few keys in the event data object, because there was a higher priority Allow parameters transformation before this one, stripping out all the keys that were not explicitly allowlisted.

Use cases

Here are some use cases for transformations.

#1 Flag custom events as conversions for GA4
#2 Enrich ecommerce data
#3 Hash sensitive URLs
#4 Populate identifiers from Firestore storage
#5 Clear unwanted parameters from GA4’s event data object
#6 Force GA4 hits to show up in DebugView
#7 Augment event with first-party user data

Naturally, all of these use cases have been doable before with server-side Google Tag Manager in one way or another. However, not all tags allow you to modify the information consumed from the event data object.

Furthermore, the problem with modifying individual tags is that you might slip and miss a tag, which then becomes a liability in your tracking schema. By nipping the problem in the bud using a transformation, the event data object itself is modified making it impossible for a tag to access something that doesn’t exist anymore.

Use case #1: Flag custom events as conversions for GA4

One of the “quirks” of Google Analytics 4 is that it determines conversions in two places: in the Google Analytics 4 Admin interface and in the client-side implementation, where conversion events are flagged with a specific URL parameter (_c=1) that tells GA4 the incoming event is, indeed, a conversion.

This second context can result in a bit of a headache if you’re using a server-side container to duplicate events across different Measurement IDs.

For example, let’s say you have a Google Analytics 4 event stream collecting to Measurement ID G-1234567. In this stream, the event click_to_call has not been flagged as a conversion, so the event requests do not have the _c=1 parameter.

However, in server-side GTM you forward this event to G-2345678 where click_to_call has been flagged as a conversion. But because the original request didn’t have the _c=1 parameter, this conversion is not registered correctly.

Augment event transformation to the rescue!

Above, I’m setting the parameter x-ga-system_properties.c to the value 1. By comparing the Event Data object with the incoming request parameters, I know that this corresponds with the URL parameter _c=1.

Additionally, I use a Lookup Table that returns true for all the events that should have this parameter added to them:

Finally, I’ve defined the transformation to only apply to the G-2345678 GA4 tag in the Server container.

Now the outgoing request to G-2345678 for any event flagged as a conversion with the Lookup Table will always include the _c=1 parameter.

Use case #2: Enrich ecommerce data

One of my favorite data enrichment use cases is for ecommerce data.

Just imagine a scenario where all you need to do client-side is collect a purchase event with a transaction_id. Then, in server-side GTM, you can use the transaction_id to pull in the rest of the ecommerce data from your sales engine (using HTTP API calls) or from Firestore, if it’s something you’ve already set up.

You can now move this logic to an Augment event transformation so that all your conversion tags can make use of this enriched data automatically.

You could even replace the value collected from the client-side hit with the actual profit, and send that to your vendors. You’d never want to expose your profit margins in the client, but in a server-side environment it’s very doable indeed.

The benefit of using this Augment event transformation is that the augmentation applies to the event data object, which means that tags that make use of it (for example the GA4 tag) won’t require any modifications in the tags themselves. They’ll just see the transformed event data object and work with the augmented values out of the box.

Use case #3: Hash sensitive URLs

Sometimes the URL itself can contain information that you don’t want to forward to a third-party vendor. However, the vendor might still need URL data for cohort building or something similar.

With a properly setup Augment event transformation, you can now hash the page_location value whenever the URL has the potential of containing sensitive information.

For example, in this case we want to use the sha256 Hasher variable template to hash the page_location value, but only when the page_category event data key has the value logged-in and only for Facebook tags.

By doing this in a transformation, we can ensure that Facebook never gets the actual page URL for sensitive pages.

Use case #4: Populate identifiers from Firestore storage

If you have the legal basis to do so, one thing you might have been using server-side tagging for is to collect and collate a user’s identity graph.

This can be as simple as a table stored in Firestore, where all the identifiers associated with a user are stored in a neat document entry.

For example, you could store the user’s login ID as the primary key, and then under that you could add all their known client/device identifiers, session IDs, and whatever other identifiers the different marketing platforms like to collect.

With an Augment event transformation, you can see what parameters are included in the event data object and then populate the rest after performing the lookup in Firestore. You’ll be able to “fill in the gaps” of your tracking schema by overwriting new or ephemeral identifiers with something more sticky.

But – please. Make sure you have a legal basis for processing all that personal data first!

Use case #5: Clear unwanted parameters from GA4’s event data object

The GA4 request seems to collect a lot of information that the GA4 client transpiles into the event data object. Not all of this information is documented!

It might be useful to look at a handful of GA4 events in server-side tagging Preview Mode to see just what event data is generated. You can then create an Exclude parameters transformation to clear all the parameters that you’re not comfortable with from the event data object.

Similarly, you might want to remove Facebook-specific parameters from GA4 tags and vice versa, if you’re using a GA4 client to produce an event data object for both GA4 and Facebook. And this applies to other marketing vendors to whom you are collecting data by piggybacking on the GA4 stream.

Use the “Some tags” setting here, as you can use it to specify the subset of tags that should consume any given transformation.

Use case #6: Force GA4 hits to show up in DebugView

Augmenting event data can help you debug your Google Analytics 4 hits, too.

For an event to surface in GA4’s DebugView, it needs to be sent with the special URL parameter _dbg=1 to GA4’s collection servers.

The GA4 Client parses this parameter into an event data key accessible through x-ga-system_properties.dbg. With an Augment Event transformation, it’s now easy to add it to any events that you want to explore through DebugView.

In this transformation, I’m setting x-ga-system_properties.dbg to 1 but only when the server-side tagging container is in Preview mode. I’m also limiting this to GA4 tags, because no other tags need to make use of this GA-specific feature.

Use case #7: Augment event with first-party user data

While I think you should be very careful with using enhanced conversions due to obvious privacy issues with handing first-party data to Google on a silver platter, it doesn’t stop me from exploring the technical fundamentals of how it works in server-side tagging.

Instead of sourcing the first-party data client-side, you can augment it into your server-side hits by using a transformation that fetches it from a suitable data source.

To populate the first-party data variables, you need to use the prefix user_data. Under that, you can set:

• user_data.email: the user’s email address.
• user_data.phone_number: the user’s phone number.
• user_data.address.0.first_name: the user’s first name.

And you can keep adding to the address object in addition to the first name with stuff like city, country, last_name, region, and postal_code.

You don’t have to hash them yourself, as they will be hashed by the Google tags before the outgoing request is compiled.

With a transformation like this, it’s absolutely vital you only enable it for vendors that you trust using the Affected tags setting. Otherwise you might be looking at a severe data breach possibility.

Summary

I’m very happy with this feature release. I think server-side Google Tag Manager is close to having a very satisfactory feature set to cater to an unbelievable number of marketing use cases.

The benefit of transformations for data governance should be obvious:

1. Google strongly recommends and incentivizes vendors to utilize the event data object when sourcing values for their tag templates.
2. With transformations, you have full control over what the event data object ends up looking like, regardless of what the client originally produces.
3. This control can be used for adding, removing, and updating values in the event data object.

Naturally, tags don’t have to use the event data object. As transformations obviously pose a risk to malicious tags that want to collect as much as they can without users having a proper say, these tags could now proceed to scrape the information from the request URL itself or from cookies or other HTTP headers.

Hopefully there will be additional UI signals in server-side Google Tag Manager that help us clearly see what data each tag template is using.

One of the problems with transformations is that you need to know both what the event data object can look like at different times and which parameters each tag makes use of.

A solid way of figuring out what a tag template is actually trying to do is to observe its permissions. Good template design involves adding as few permissions as possible, so if a template clearly lists which event data keys it wants to access, you can build your transformations more efficiently.

Obviously templates can also be built with wildcard permissions access, in which case this sleuth work won’t be as effective. I hope that Google figures out a way to incentivize limited permission sets over broader wildcard selections.

A few features that I’d still like to see in transformations include:

• Ability to select a tag type over individual tags when choosing Affected tags.
• Ability to populate the table of parameters with a single variable, so that you could iterate over all the keys in the event data object, for example, and return an array (for Allow / Exclude) or object (for Augment) of parameter names (and values) to which you want the transformation to apply.

What do you think about transformations? Do you have additional use cases in mind for them?