Go channels at horizontal scale (powered by message queues)

Go channels at horizontal scale

• Use Go channels transparently over a messaging queue technology of your choice (Currently NATS, Redis or NSQ, Amazon SQS)
• Swap vice.Transport to change underlying queueing technologies transparently
• Write idiomatic Go code instead of learning queue specific APIs
• Develop against in-memory implementation before putting it into the wild
• Independent unit tests (no need for running queue technology)

Usage

This code receives names on the |names| queue, and sends greetings on the |greetings| queue:

// get a Go channel that will receive messages on the
// |names| queue
names := transport.Receive("names")

// get a Go channel that will send messages on the
// |greetings| queue
greetings := transport.Send("greetings")

// respond to |names| messages with |greetings|
for name := range names {
	greetings <- []byte("Hello " + string(name))
}

• The code above is illustrative, be sure to read the design patterns
• Always stop the Transport, some technologies register and deregister their interest in the queues (this means trapping signals and gracefully shutting down services before exiting)
• Use Send and Receive methods to get channels, which you can then use as normal
• Be sure to always handle the ErrChan() error channel to make sure the underlying queue technology is healthy

Quick start guide

• Write your services with unit tests using normal Go channels (see our design patterns)
• Install Vice with go get github.com/matryer/vice/...
• Select a messaging queue technology
• Build a command to run your service

Read the blog post: Introducing vice: Go channels across many machines

Acknowledgements

Special thanks go to David Hernandez, Jason Hancock and Piotr Rojek for their support on this project.