Helping a Large Insurance Company Pave the Way for GA4 with AWS
Quick Take
In large organizations siloed technology and IT functions rarely have a synergistic relationship as competing priorities present major challenges, often resulting in conflicting efforts and goals. Enterprise IT stretched thin serves the entire company maintaining from maintaining its infrastructure and security to keeping the lights on. IT groups run by the business units specialize in analytics and data that serve that specific group’s needs to maximize their units’ goals. For a large life insurance company, Merkle was able to bring the organization together to build a solution that fully integrates with the new Google Analytics 4 (GA4) service and other sources to successfully share data across the business.
Challenge
The marketing technology team at the large life insurance company had multiple channels of advertising spend on various platforms with no insight or segmentation to the data. The martech team had an array of disparate sources of information within Google Analytics, Big Query, MS SQL Server, and various SaaS Platforms including Mainframe without a place to house that data to bring all these key data points together in a repeatable and scalable solution. A suite of powerful tools on AWS would be the best way to build an effective data pipeline that could extract from numerous sources, clean it to make understandable, and then activate from it. The client’s recruiting team also wanted to understand how their dollars were best being spent and if tactics were effective in attracting new talent while their current platform, Google Analytics was migrating from GA360 to GA4.
Solution
The insurer chose AWS because it is a comprehensive and broadly adopted cloud platform, offering over 175 fully featured services from data centers globally. Millions of customers, including fast-growing startups, large enterprises, and leading government agencies use AWS to lower costs, become more agile and innovate faster. Brands choose AWS for their open, scalable, and cost-effective cloud solutions for enabling real-time analytics and data streaming technology that enables users to ingest, process, and analyze large volumes of high-velocity data from a variety of sources in real time.
Simple and Effective Solutions
Merkle working with Enterprise IT and the development patterns established were able to build an effective data pipeline that enabled innovation and delivery for the martech team to ingest and understand all the different channels to build pipelines to clean and activate opportunities to scale growth by:
- Building a datamart on S3 for the martech team
- Using AWS Transfer and Lambda functions to ingest data from all of the disparate sources and establishing a pattern for additional sources for the future
- Cleaning the structured and unstructured data with Glue for a proven manner to best understand it
- Building GA4 extracts by leveraging Glue BigQuery connector and Lambda functions, executing it using Step Functions
- Creating Google Analytics table on Athena leveraging a complex partitioning strategy to keep all accounts under one object for easier data access and analysis
- Deploying the entire serverless solution using CDK in a Dev/Ops pipeline for automated deployments and faster development
- Using Athena and RedShift to analyze the various campaigns
- Enabling data sharing to other units within the business to build data lakes
Results
Created a data pipeline sourcing various sources of data from Zoom, mainframe, SQL Server, Google Analytics, and Salesforce to the martech teams’ data mart. The solution was secured, automated, and self-dependent allowing all teams to focus on their priorities.
Allowed client’s marketing team to understand and scale their spend across channels to recruit new agents to generate more revenue.
Established a scalable framework as additional data sources became available and other organizations within the business-like Actuaries to accelerate their data pipelines by leveraging automation.
