RabbitMQ

An Open-Source Implementation of AMQP in Erlang/OTP




www.rabbitmq.com
written by ourselves, i.e. LShift, in a joint venture with CohesiveFT

The Advanced Message Queueing Protocol


The Open-Standard Protocol
for Business Messaging

started: 2004, first version: June 2006, latest version: February 2008

www.amqp.org

objective best illustrated by contrasting with another form of messaging: email

Everyone uses email without thinking, so why is commercial business messaging - getting applications to talk to each other - so hard?

With email we have ...

  • ubiquitous substrate - email infrastructure is part of the internet fabric; it's supported by backbone, ISPs, your corp IT departments
  • products - email clients, networking kit, spam filters
  • applications - integral to many solutions (e.g. stuff LShift have built in web space); web forms, monitoring, bug trackers, ordering systems

We have none of these in business messaging. Instead one has to undertake a development and integration project every time you want to make two applications talk to each other, not to mention trying to do that across organisational boundaries.

Imagine what it would be like to use email if you had to do undertake a major development project every time you wanted to send an email to an organisation that you hadn't dealt with before.

Yet this is exactly the situation we are in with business messaging!

Essentially what has been missing is an open, standard protocol that caters for the bulk of business messaging use cases and gets widely adopted.

AMQP aims to be exactly that.

in development for ...

more information at ...

AMQP Working Group

AMQP is governed by the working group, consisting of two kinds of members...
  • vendors that produce products which implement AMQP (more about what exactly that means in a second)
  • two groups of users - end users who develop and deploy AMQP-based solutions and OEMs who embed/utilise AMQP in their products and solutions.
  • plus some feedback from wider community

AMQP Model

on to the technical bit...

Model - AMQP's take on the world of business messaging

  • the server (aka broker)
  • producer
  • exchange
  • binding - routing based on x type and binding
  • queue - where messages live until consumed, possibly persisted
  • consumers - pull vs subscribe & push
Intermediation is an important principle in AMQP that distinguishes it from other messaging models - this is not P2P or RPC.

The 'P' in AMQP

The P in AMQP

  • producers and consumers are logically part of AMQP clients
  • which talk to AMQP server
  • over the P in AMQP

Model and Protocol

introducing two more concepts: connections and channels
  • connections - tcp connections
  • channels - multiplexing logical streams of commands and data
  • channels are boundary between model and protocol
  • soup of exchanges, queues and bindings - i.e. the routing set up, management of which is part of protocol - unlike email

RabbitMQ History and Objectives

  • first AMQP implementation to be written from scratch from the spec - all others were written by folks involved in the initial specification process and during that process
  • thus RabbitMQ validates the spec; proof that it can be implemented and interop can be achieved
  • scalable - in several different ways, multi-core, cluster (more about that later) but also size of hardware - phone, slug - 16-way Intel.
  • extensions are one area where RabbitMQ really shines ...see next slide

RabbitMQ Universe

  • native AMQP handler
  • transports - extensions of the RabbitMQ server that expose the AMQP protocol over transports other than raw TCP/IP. Sometimes accompanied by client libraries.
  • gateways - extensions of the RabbitMQ server that bridge between AMQP and various other messaging networks
  • clients - These conform to the client role of the AMQP protocol and generally interoperate with other AMQP brokers. light blue - developed for RabbitMQ, but work with other brokers, pink bits - developed by community, not necessarily for RabbitMQ
  • dual role of Erlang client
  • adapters - These adapt various messaging abstraction, most of them ESBs, to AMQP. They are typically implemented on top of one of the AMQP client libraries, but they do not expose the AMQP functionality directly.
Rest of talk is mainly going to focus on the server and extending it with a new transport; so just bear in mind that there is a lot more to RabbitMQ than the server.

Why Erlang/OTP?


protocol handling is an Erlang/OTP sweet spot

Erlang Development Tools

  • Another main reason why we chose Erlang/OTP is the excellent development tools it provides. These are going to be the main subject of this talk.
  • with just one or two exceptions all of these ship with OTP
  • we are going to give a glimpse of many, but not all, of these in the following section

IDEs

vim, emacs, erlide + distel, wrangler
All the usual stuff people have come to expect from an IDE.


Action!

Erlang Development Tools - Summary

Erlang/OTP is a mature environment

just a quick recap of what we've seen


Action!



Questions?