Streaming Real-Time Sensor Data with Grafana Live and MQTT

Introduction

In robotics and automation, a common requirement is to capture and visualize real-time data from hardware components such as sensors and actuators, which helps understand overall system behavior, diagnose potential issues, and reduce overtime.

Historically this required complex tools and extensive custom development to stream data into typical analysis and visualization tools (e.g., Grafana). With the release of Grafana 8.0, the new Grafana Live streaming API makes streaming and visualizing real-time data much easier.

(Note: “real-time” refers to “soft real-time”; network latency, garbage-collection pauses, and protocol limits can cause delays of several hundred milliseconds.)

This article explores the use of Grafana Live’s streaming API in the context of a sensor system that communicates via the MQTT messaging protocol.

Setup

MQTT is a lightweight publish-subscribe network protocol commonly used for message passing between devices over TCP/IP. It was originally developed by IBM and is widely used in bandwidth-constrained environments for machine-to-machine communication where compact data transmission is essential.

Grafana Live is an integrated real-time messaging engine built into Grafana, introduced in version 8.0. It is based on a publish-subscribe framework that communicates over the WebSocket protocol and allows any type of event data to be pushed to front-end clients instantly.

To demonstrate a realistic scenario, we use a simple IMU-based attitude sensor board connected to a Raspberry Pi, transmitting accelerometer and gyroscope data via MQTT.

The sensor board uses Bosch Sensortec’s widely adopted BMI160 6-DOF IMU, a compact, low-power, low-noise device commonly found in smartphones and wearables. The board connects to the Raspberry Pi via I²C; a Python script reads the sensor data and periodically publishes it to a locally running Mosquitto MQTT broker (alternatively, the open-source mica-mqtt broker can be used).

To stream the sensor data into a Grafana instance on a separate machine, we install and use the Grafana MQTT datasource plugin on that machine. The plugin’s integrated MQTT client subscribes to MQTT topics and publishes them to Grafana via the Grafana Live Streaming API. The datasource is configured to connect to the Mosquitto broker running on the Raspberry Pi.

For the dashboard we use the time-series panel (available in the v7.4 preview) to visualize each axis of accelerometer and gyroscope data, and the newly released timeline panel to detect motion on any of the three axes. These visualizations are useful for systems where abnormal or unbalanced behavior appears as excessive movement or rotation around an axis, and they can be critical for monitoring dynamic systems with complex attitude-control mechanisms.

Demo

The final live demonstration shows how the time-series trends respond to changes in direction and rotation around all three axes of the sensor board, and how the state timeline is useful for independently detecting motion on any axis.

Conclusion

This simple application demonstrates the power of Grafana’s new streaming API, especially when combined with various visualizations that run natively. It opens the door for streaming data from a wide range of communication protocols and hardware platforms through Grafana’s flexible plugin mechanism.

If you are interested in implementing streaming functionality for your own applications, be sure to consult the reference guide to develop your own streaming backend datasource plugin.