The Wallaroo Labs
Our technology helped a top-notch adtech engineering team deliver real-time analytics on 50MM ad bids per second using 20% of the infrastructure. Check out more below.
What is Wallaroo?
Wallaroo combines an efficient, stateful compute engine with on-demand scaling, together with data connectors to easily plug into your data ecosystem. It's designed for large-scale analytics, streaming data, & AI applications. It can run on as little as 10% of the infrastructure and provide up to 100X speed-up (vs. Apache open-source projects) for use cases dealing with immense amounts of data.
Wallaroo takes care of the annoying stuff out of the box:
Distributed state management
A large ad exchange analyzed bid stream data on a nightly basis to optimize their auction algorithm. Their data science team discovered that the insights they were generating lost significant value over time, and were most effective when fed into their algorithm as soon as possible. To take advantage of this they decided to move their analysis from batch to real-time provided they could keep costs down and maintain accuracy.
The Brick Wall
The engineers built prototypes with existing open source tools but found they couldn't keep up with the hundreds of terabytes per day (millions of events a second) they were trying to process. Worse still, they were losing messages when the system was overwhelmed, and since they would need to deploy on a lot of servers, complexity became a serious concern. The engineers started to worry that it was going to be impossible to meet the business goals while maintaining accuracy and adhering to the budget.
That's when Wallaroo Labs got the call. We worked with the client's tech team to determine the goals and SLAs for the project. After a handful of meetings we proposed an initial prototype using our distributed computing technology. In 6 weeks, we had a proof of concept using Wallaroo that was beating their original volume and performance goals, and keeping things well within their budget. 4 weeks later, the system was in production. Wallaroo had made it possible.
Solution running on 20% of expected infrastructure saving $800k per year
Analyzing 50MM bids per second with no data dropped over rigorous accuracy tests
50% time to launch compared to initial DIY projection
In Their Words
"Wallaroo is absolutely well-timed for a wide variety of companies... too often tech companies reinvent the wheel on core data infrastructure, distracting them from focusing on competitively differentiating products. Wallaroo exceeded our expectations and performance standards" - Client CTO