How can the Modern Data Stack help teams overcome the limitations of GA4/GO sunset?
This interview is part of Kameleoon's Expert FAQs series, where we interview leaders in data-driven CX optimization and experimentation. Timo Dechau is a Product and AWS Architect for internal services at Audible and the founder, Tracking & Analytics Engineer at Deepsky Data.
What is the Modern Data Stack (MDS)?
The Modern Data Stack (MDS) is a data stack. We’ll talk about its significance in a second.
First, let’s get the “modern” bit out of the way. The “modern” data stack is foremost a marketing invention. It was introduced to set specific tools apart from existing ones. Something that always happens. But there was a difference between these data stacks and the ones before. Let’s have a look at what makes the difference.
Pre-MDS data setups were either built with an integrated data platform (which was very expensive and needed a bunch of training for a proprietary setup) or were built with open-source technology that took teams and plenty of experience to maintain them. Ultimately, it meant that data stacks were exclusive to just some companies.
The MDS changed this. It made the data stack affordable and accessible for everyone. The first step, and basically the door opener, was new types of databases. Today, we call them cloud warehouses, starting with AWS Redshift and BigQuery, followed by Snowflake. They enabled affordable setups for massive data storage and quick query times.
Next came the standard data integration tools. Data integration was a custom job. You build a pipeline to extract data from an API, transform it, and load it into storage. Building and maintaining these took a lot of time and resources. Tools like Stitch or Fivetran offered standard pipelines for services like Google Ads, Facebook Ads, or Google Analytics and made data integration a one-click problem (at least for supported connections).
After that, we saw a wave of new tools that helped to make loading, transforming, and working with the data more accessible—one of the trends was moving from ETL (Extract, Transform, and Load) to ELT, with the transformation being performed within the data warehouse. This aspect of the MDS allows businesses to take a best-of-breed approach. You can pick the tools that work best for you from the hundreds available and switch them if something better comes along. I know it sounds like paradise, but truth be told, it is not that easy.
Today, we can see a phase that some people call the post-modern data stack (again, marketing to set something apart). The MDS has parts that need some evolution, which is happening. But in general, MDS made working with data stacks accessible for more companies than before.
Companies today can load data from different sources like their analytics tools, ad platforms (like Google Ads and Meta), testing tools, CRMs (like Hubspot or Salesforce), backend databases, and many more into a cloud analytical database like Snowflake. From there, you can build a data model that combines these different sources into extensive customer data, applying things like lead scoring, offline conversions, or churn probabilities. Then, load this data back into the marketing or sales system to create better-targeted activities. The MDS also enhances the product data available to teams, giving them a deeper understanding of how features perform.
What benefits does the MDS offer product experimentation teams?
With the MDS, teams can now use data from different sources to enhance their experimentation data.
Software teams might already use feature flag systems to manage new feature rollouts. You can load this data into the data warehouse and combine it with behavioral and commercial data to analyze experiments without additional tracking.
Before the MDS, it was tough to get product metrics, but now teams can release a new feature under a feature flag and track the revenue generated from it, the impact on the performance of the user's device, load on the server, crash rate, etc. On top of this, thanks to a customer 360° view, product managers can monitor the long-term financial impact, such as customer lifetime value tied to a flag.
It also allows you to create deep segments for experimentation. Segments before relied on tracking data about the user. With the MDS, you can enrich the user data with data coming from CRMs, Customer Success, or Customer support tools. This gives you new ways to design experiments with deeper segments.
What are the pros and cons of building your MDS using Google applications such as GA4, BigQuery, and Looker?
The big pro is that you do everything on The Google Cloud platform. As pointed out before, the MDS is famous for saying that you can combine all different tools. However, this can create significant overhead for integration work. And all people working on the stack have to switch between tools constantly.
Building an MDS on one platform, like the Google Cloud platform, benefits you with easier integration between the different tools and just one learning platform. Google Analytics 4 data loads into BigQuery with one checkbox in the settings. You can use Dataform for free to add all your transformations and apply them to your Google Analytics 4 data. Then, use the Looker and BigQuery connection (ready with a few clicks) to access the data and make it accessible to your company. Staying on the Google Cloud platform should get quicker results that are valuable for your company.
But if you want to incorporate different tools because you want a particular tool next to Looker, you need to check how easily it integrates with the Google stack. Another thing to keep in mind is you still need to learn and master the platform. You get the Google Analytics 4 data with just one click, but this is just the first step. You need to know how to model the data, work with SQL, make the setup performant, and not burn too much money to get real insights from the data.
How can experimentation teams design their MDS to overcome the limitations of Google data/analytics applications and the GO sunset?
While experimentation teams can still use their Google Analytics 4 (GA4) data, they will need to find an alternative testing tool to Google Optimize to launch and measure their A/B tests. The benefit of sticking with GA4 is that you can still use your Google Analytics data and don’t need to start to track events twice for a different system.
If your business plans to use GA4, data will need to be exported into BigQuery and passed downstream to something like Looker for deeper analysis (which requires SQL). In this case, you’ll want to look for testing tools that integrate with BigQuery. However, there are limitations to sticking with the Google ecosystem, and some companies might opt to build their stack around other tools, such as Segment + Mixpanel + Snowflake or Snowplow + Airbyte + Spark.
Whatever toolset you choose, the MDS setup should allow you to view the experiment event data and essential conversion events, combined with a host of other data sources from product and user analytics to help you better investigate test results and user behavior.
How can the MDS be designed to help democratize data usage among product and experimentation teams and the wider organization?
This starts with feature flagging. When feature flags are essential to each deployment, you can control the rollout and have the technical setup for creating new experiments quickly. This system also gives vital data on users entering an experiment and which variant. Combined with Google Analytics (or other) event data, it provides the foundation to analyze experiments.
The second step is a solid framework to analyze the experiment data based on statistical significance and similar success metrics. This way, each experiment is analyzed and valued based on the same framework, making them comparable.
All of this needs to be accessible to all teams. This requires a user interface to set up new experiments (or features), analyze them, and make it easy to share the results. The MDS allows teams to combine test, analytics, and product data into business Intelligence tools so that each team can have customized dashboards based on a single source of data stored in the data warehouse.
Timo, can you share more about your career journey and how you combined your love of product and data?
I started in product and stayed there for over eight years. One thing I was struggling with in the beginning was getting heard by my stakeholders. They were senior management and ruling the backlog. I was a project manager paid to get it done. Data helped me change the conversation. On the one hand, by retroactive analysis, and on the other, by running changes with experiments. That changed my impact and the quality of the product.
After that, data setups with event data became a must-have for me, and at some point, I wanted to focus only on data. So, I switched roles from product to data.
Product analytics and working with event data are still my focus today. It is mainly because it is just a big puzzle, and you don’t get a picture of what it should look like. Working with this kind of data requires a lot of design thinking and exploration. So it is never boring.
I still do hands-on setups, but my focus is shifting to education. I have a live show (Better Together), a YouTube channel, and a Substack where I share my thoughts and ideas about product analytics, event data, and how you can use both to build better products.
