Cloud, IoT, and AI will drive 10X more data volume, 10X faster data, and an explosion of complex business algorithms that could drive processing needs by 50X in just five years.
Wallaroo was made with one simple mission: To democratize data processing by removing the barriers of complexity, cost, performance, and language support. It's what a modern data processing engine should be.
Scale-Independence is the magic in much of what we do, and it's how we make the infrastructure virtually disappear. Scale-Independence means that you can scale without changing code. With Wallaroo, it means you write to Wallaroo's scale-independent API and then run your code on Wallaroo, across any number of servers whether on-premise or in the cloud. Your business logic is independent of scale and infrastructure, while Wallaroo is aware of the scale and infrastructure and takes care of all the "plumbing" for you.
Scale-independence allows you to go from a developer's laptop to testing to production without having to change any code. The code you run on your laptop in a single process is the same code you run in production across 100 processes.LEARN MORE
Simple Native APIs
Write your application in Python or Go using the libraries you know and love. The Wallaroo Python runner embeds the Python 2.7 interpreter (3.x coming very soon) and works with standard tooling like virtualenv.
Compiled languages like C++ and Go create standard binaries. No need to set up the JVM and do complex marshaling to and from Java. Use your language with the tools and libraries you already know.
Got a machine learning algo in Python? Are you using Matplotlib, NumPy or Pandas? We've got you covered. No more developing in one language and converting to another to run in production.
Our simple, native APIs are great because there’s no Java, no JVM. No muss, no fuss.
Managing distributed application state is hard, but it doesn't have to be that way. Wallaroo's scale agnostic developer APIs present a simple interface for working with state. Safely operate on your application state regardless of scale. The same code you use to query and update your application state in a single thread works just as well when you use a massive parallel cluster.
Wallaroo's scale aware platform handles the complexity of distributing in-memory state across a cluster of machines. Turn on resilience mode, and you'll be able to recover state after a crash, without ever changing code.LEARN MORE
Add workers to and remove workers from a running cluster without having to restart. No longer able to handle your workload with three machines? Add more! Scaled up but don't need the capacity anymore? Shrink the cluster! Wallaroo will automatically repartition your application state to match available capacity, without having to restart.LEARN MORE
Wallaroo is a lean, mean, data processing machine. We've designed the engine from the ground up to utilize your resources efficiently. That means you can do the same amount of processing while using fewer resources! Wallaroo applications are capable of microsecond latencies. One of our Python applications can process over 80,000 messages per second on a single CPU. And all 99.99% of the messages are handled in 4 milliseconds or less.
What's our secret sauce? Wallaroo’s performance comes from a combination of design choices and constant vigilance. Wallaroo uses an Actor model approach that encapsulates data, minimizes coordination, and keeps application state close to the computation. We continuously test every feature for performance and reimplement functionality when it's not up to speed. Looking for performance? You're looking for Wallaroo!LEARN MORE
Exactly-Once & Resiliency
Wallaroo is built to get you results quickly and accurately. Wallaroo's resilience features plus with our integrated state management allow you to feel as confident in your results after a disaster as you do on a typical day. Wallaroo manages the resiliency of your state for you. No messing with external databases to maintain your application state. No worrying that it will recover correctly after a failure.
Wallaroo writes all state changes to an event-log that is used to reconstruct your state after a crash. Avoid lost or duplicated messages with our exactly-once message processing features.LEARN MORE
Wallaroo is built on a high-performance actor-model runtime. Our approach helps encapsulate data, minimize coordination, and bring state close to the computation. Actors allow us to model our core concepts easily and accurately, which translates into a fast and solid framework for building your applications.
Performance is a core concern for us. We built a system to measure the end-to-end latency of each message processed, as well as the time spent in individual process steps. That's so we could test every feature added to Wallaroo and eliminate as much overhead as possible.
You can use the same Metrics system and UI to easily see how your own application is performing. The Metrics UI is a tool to allow you to get live metric updates for any connected Wallaroo application. The Metrics UI will provide you with a general and detailed performance overview of each running application.
Using the metrics UI, you can see latency and throughput metrics for your running application in real-time, broken down by application, pipeline, worker, and computation. You can also integrate the live stream of metrics data in your reports or data repository.
Stay informed on Wallaroo Labs news, software releases, and best practices
Wallaroo makes the infrastructure virtually disappear so you get rapid deployment, very low operating cost, and elastic capacity with zero downtime for your applications in big data, stream processing, machine learning, and microservices.