WallarooLabs / wallaroo
- пятница, 29 сентября 2017 г. в 03:16:13
Ultrafast and elastic data processing
Wallaroo is a fast, elastic data processing engine that rapidly takes you from prototype to production by eliminating infrastructure complexity.
Wallaroo is a fast and elastic data processing engine that rapidly takes you from prototype to production by making the infrastructure virtually disappear. We’ve designed it to handle demanding high-throughput, low-latency tasks where the accuracy of results is essential. Wallaroo takes care of mechanics of scaling, resilience, state management, and message delivery. We've designed Wallaroo to make it easy scale applications with no code changes, and allow programmers to focus on business logic. You can learn more about Wallaroo from our "Hello Wallaroo!" blog post.
We've done a 15-minute video of our engineering presentation that has helped people understand what Wallaroo is. If you watch it, you will get:
Existing language bindings:
Planned Language bindings:
Please see status for language binding support details. Wallaroo is open source software with an expanding software community. Please see the How to Contribute section if you wish to help support your favorite data analysis language.
Wallaroo is open source software with an expanding software community. Please see the How to Contribute section if you wish to help support your favorite operating system.
Wallaroo applications are user hosted. It's equally at home "in the cloud" or "on-premise."
We have no "as a service" offering at this time.
Interested in where we are taking Wallaroo? Check out our roadmap.
Wallaroo is primarily written in Pony. As such, Pony is the first language to receive support for any given feature. We don't expect the Pony API to get much usage outside of Wallaroo Labs. We aren't maintaining any documentation for the Pony API outside of a few examples. You are welcome to use the Pony API but are going to mostly be on your own documentation wise.
Currently, Python 2.7 is our primary focus. As we add features to the Wallaroo, we will be adding corresponding Python APIs and documentation.
We are currently working with a client who needs Python 3 bindings. We plan to introduce Python 3 bindings by the end of 2017.
C++ was our first non-Pony API. Since that time we have learned a lot about writing Wallaroo language bindings. We plan on revisiting the C++ API in the near future to improve its ergonomics. If you are interested in using Wallaroo with C++, you should contact us. We're happy to work with you.
We are currently working with a client who needs Go bindings. We plan to introduce Go bindings by the end of 2017.
JavaScript support is currently in the planning stages with a release in 2018.
We have numerous issues open to improve existing Wallaroo functionality. For a high-level overview, please see our current limitations document.
Are you the sort who just wants to get going? Dive right into our documentation then! It will get you up and running with Wallaroo.
Our primary documentation is hosted by GitBook at http://docs.wallaroolabs.com. You can find additional information on our community site.
We're an open source project and welcome contributions. Trying to figure out how to get started? Drop us a line on IRC or the developer mailing list, and we can get you started.
Be sure to check out our contributors guide before you get started.
Wallaroo is an open source project. All of the source code is available to you. Most of the Wallaroo code base is available under the Apache License, version 2. However, not all of the Wallaroo source code is a[Apache 2apache 2 license licensed. Parts of Wallaroo are licensed under the Wallaroo Community License Agreement. Source files in this repository have a header indicating which license they are under.
The core stream processing engine and state management facilities are all licensed under the the Apache version 2. Autoscaling, exactly-once message processing and resiliency features are licensed under the Wallaroo Community License Agreement.
The Wallaroo Community License is based on Apache version 2. However, you should read it for yourself. Here we provide a summary of the main points of the Wallaroo Community License Agreement.
Please contact us if you have any questions about licensing or Wallaroo Enterprise.
15 minute overview of key Wallaroo features. Includes actual code and a demonstration of our stateful autoscaling functionality.
Our open source annoucement.
An introduction to Wallaroo.
A look inside Wallaroo's excellent performance
The company behind Wallaroo.
Wallaroo documentation.
Our VP of Engineering Sean T. Allen talks about one of the techniques we use to test Wallaroo.
Wallaroo currently exists as a mono-repo. All the source that makes Wallaroo go is in this repo. Let's take a quick walk through what you'll find in each top-level directory:
Markdown source used to build http://docs.wallaroolabs.com. http://docs.wallaroolabs.com gets built from the latest commit to the release branch.
Code for writing Wallaroo applications using C++. This is currently unsupported.
Wallaroo example applications in a variety of languages. Currently, only the Python API examples are supported. See status section for details.
TCP utility applications that can stream data over TCP to Wallaroo applications and receive TCP streams from said applications.
The Pony source code that makes up Wallaroo.
Python runner application. Machida embeds a Python interpreter inside a native Wallaroo binary and allows you to run applications using the Wallaroo Python API.
Source for the Wallaroo metrics UI.
Tools we use to create machines in AWS and other environments.
Tools we have written that are used to test Wallaroo.
End user utilities designed to make it easier to do various Wallaroo tasks like cleanly shut down a cluster.