Criteo is in the process of migrating to Aerospike Database to reduce the number of servers it needs to power its global advertising marketplace from 5,000 to 600.
Powering a global advertising network is no small task, requiring computing capacity around the world, as well as a database layer that is able to quickly respond to queries.
That's the challenge that faces Maxime Brugidou, engineering director of platforms at Criteo, who is tasked with helping to build out the database layer at the company.
Headquartered in Paris, Criteo is publicly traded on NASDAQ under the symbol CRTO and provides a global network where organizations bid in real time for online display ads.
Criteo runs multiple types of databases, including relational databases as well as a key-value store database layer, which powers the company's online bidding system for advertisements, Brugidou explained.
Maxime Brugidou
To date, Criteo has been using a large global deployment with on-premises data centers in Asia, Europe and North America, with approximately 5,000 servers. The key-value store database layer was using a number of different technologies, including the open source memcached caching technology, to help accelerate query response time.
Starting in July 2021, Criteo began moving to a new system, powered by Aerospike Database, that will reduce the need for a caching layer and enable a real-time database deployment. Criteo expects that it will only need 600 servers with Aerospike to power its global needs.
The challenge of powering a real-time database at Criteo
Criteo needed a fast, efficient key-value store database to power its application on which organizations bid for ads in real time.
We have a lot of real-time decisions to make involving a lot of data and we need sub-millisecond processing reading and writing data all over the world.
Maxime Brugidou Engineering director of platforms, Criteo
"We have a lot of real-time decisions to make involving a lot of data and we need sub-millisecond processing reading and writing data all over the world," Brugidou said.
A key challenge Criteo faced in enabling the key-value store data layer was that the company needed to use too many computing resources to effectively enable the performance it needed. The Criteo system must be able to respond to up to 200 million queries per second during peak times.
To address the problem, Brugidou and his team began to look for and evaluate different technologies and found Aerospike.
Aerospike is a real-time database technology that enables a persistent data caching layer using NVMe-based storage. Brugidou noted that without Aerospike, Criteo's system largely relied on RAM system memory.
The key advantage of the Aerospike approach is better performance with fewer servers, as more NVMe can be placed on each server than actual RAM, he said.
Brugidou said the move to Aerospike is a continuing process, involving a hardware refresh across Criteo's global data center, and it will support up to 300 TB of data when complete.
Maxime Brugidou
