MQTT as a Unified Message Bus for Infrastructure Services

Presented by Matthew Treinish
Tuesday 4 p.m.–4:15 p.m. in Great Hall CB01.05.009
Target audience: User

Abstract

Development and testing of the OpenStack project operates at a tremendous scale, with hundreds of code repositories and thousands of contributors interacting continuously. The infrastructure to support this has to operate at an equally large scale to ensure that it is not outpaced by the volume of upstream development activity. Enabling users and other consumers to see what is happening in real time in this increasingly complex infrastructure becomes equally complex and large. This is why we need interfaces available to develop tooling to handle this. MQTT best known for it's use in IoT and sensor network applications also provides a number of advantages when used as an event bus for infrastructure services. For the OpenStack developer and community infrastructure we introduced firehose.openstack.org to provide a unified message bus. Built using MQTT and Mosquitto, firehose is an interface where services can publish events. This talk will cover how you can use MQTT as a unified event bus for infrastructure services. It will explain some basics of the MQTT protocol and why it's well suited for this application. It will also use the OpenStack community infrastructure's firehose as a case study to explain the benefits and how a similar system can be used for their your own needs, experimentation and innovation.

Presented by

Matthew Treinish

Matthew has been working on and contributing to Open Source software for most of his career. He has been primarily contributing to OpenStack since 2012 and is a former member of the OpenStack TC (Technical Committee) and was previously the PTL (project technical lead) of the OpenStack community's QA program from OpenStack's Juno development cycle in 2014 through the Mitaka development cycle in 2016. He is a core contributor for several OpenStack projects and a member of the OpenStack Stable Maintenance Team. Matthew currently works for IBM's Developer Advocacy team working to make Open Source software better for everyone. He has previously been a speaker at OpenStack summits, LinuxCons Japan, China, and North America, OpenWest, FOSSASIA, PyConAU's OpenStack miniconf.