Holiday Data Spikes and Acquia CDP Platform Scalability

A key differentiator of the Acquia customer data platform (CDP) is its scalability. Scalability is one of its key architectural tenets, powered by our tech stack’s horizontal scalability and elasticity. You see this in how large, high-demand enterprise brands deliver relevant and personalized customer experiences via the Acquia CDP, processing hundreds of millions of customer records and billions of transactions and events daily. One illustration of Acquia CDP’s scalability is how easily we can add enormous volumes of historical data into the platform when onboarding each new enterprise client. 

You can also observe Acquia CDP’s scalability in its ability to handle and ingest big spikes of data as we did during this Black Friday and Cyber Monday for our retail clients. Let’s dive further into some interesting aspects of these data spikes. 

The CDP real-time pipeline ingests real-time events, including web events (browse as well as checkout/transactional events) and email events. We saw huge spikes across brands in these events and transactions, and our platform successfully ingested the data spikes. This success is not just about data ingestion but also about scalability in all the other modules like campaigns, analytics, and machine learning, which all need to operate on these increased volume spikes. Supporting spikes like this is mission critical to our clients’ bottom lines.   

Data Points in Web Traffic Volumes: 2019 Black Friday / Cyber Monday

• Multiple brands had spikes of greater than 20 times their regular web traffic volume on Black Friday and again on Cyber Monday. These peaks correlate to the times when they sent large campaigns. Acquia CDP also ingested these spikes in email events, so the campaigns had double the impact on the data ingested into Acquia CDP.
• One brand had spikes of greater than 30,000 events per minute (compared to their regular volume of less than 1,000). This was early morning on Black Friday.
• Another brand sustained spikes over time resulting in greater than 1.3 million events per hour and greater than 25 million events per day on Black Friday.
• Another company normally averages 20,000 calls per hour, but on the day before Cyber Monday, it reached peaks of ~430K calls per hour, i.e., greater than 20 times its average.
• Even some of our European clients saw about 20 times their normal volume consistently throughout the holiday weekend.
• In that week, across brands, we processed more than 350 million web events — more calls in that week than an average month.

Data Points from Email Traffic Volumes

We saw spikes in email campaigns and the events they generated, which were all ingested into Acquia CDP via our extensive ESP connectors. Some interesting data points are:

• We ingested ~3.7 billion email events (which include email send, email open, email click) across all brands during the holiday month. That's 15% more than the email events in the previous month.
• We've consistently seen higher email click rates, leading to better engagement during the holidays in 2018 and 2019. For example, in 2019, while 29% of the monthly email sends were in the holiday week, 32% of the monthly email clicks were during the holiday week, showing greater engagement during holidays.
• We supported multiple intra-day spikes for web and email events on Black Friday and Cyber Monday. 

Data Points from Transactional Volumes

As expected with the increased web and email activity, there were spikes in transactional data as well. We noticed interesting data points in the transactional data ingestion:

• On Black Friday 2019, on average across brands, we ingested approximately six times our clients’ normal daily transactional volume across online and offline transactions. 
• Some brands had greater than nine times their normal transactional volume on this day.
• On average, our clients saw 13% growth in transactional volume when compared to the same period last year. A majority of that growth came from digital sales channels driven by convenient features such as "Buy Online; Pickup in Store."
• The percentage of digital transactions in the total transaction volume during the week of Black Friday and Cyber Monday was 23% in 2018, then jumped up to 30% in 2019.

Powering Machine Learning, Campaigns, and Analytics

Once these data spikes are ingested through the Acquia CDP data pipeline, the data powers all the CDP modules, illustrating these modules' scalability. So, these data spikes directly translate to helping marketers achieve better results.

• Acquia CDPs Machine Learning module generates greater than 1 billion predictions across brands
• Acquia CDP's Metrics module powers dashboards and ad hoc reports to deliver insights based on this increased event and transaction volume.
• Acquia CDP’s Actions module scales up to help marketers act on these newly ingested data spikes to deliver campaigns.