Arduino – Logging Time Series Data to Graphite

Adafruit project stores temperature and humidity readings from the DHTXX sensors to a Google docs spreadsheet. The following quick project extends the storage to use Graphite. Graphite is a scalable real-time graphing system for collecting, storing, and visualizing time series data.  I use Graphite to analyze metrics for a couple application clusters at work. Graphite is overkill for this small of a problem. Consider this a learning exercise.

Install Graphite

Installing and configuration Graphite is the bulk of this project. I suggest using a VM to isolate and simplify the setup. Ubuntu 12.04 server is a good candidate for running Graphite due to the gists available for installation. I found the following gist to be helpful when starting from a new Ubuntu install: https://gist.github.com/bhang/2703599

Build the Circuit

I followed this Adafruit project for connecting the DHTXX sensor and having my Arduino output the humidity and temperature readings over the usb serial connection.

Logging to Graphite

I’m using a BeagleBoard-xM to read the Arduino serial output and send the data to the Graphite server. You could use any Linux system capable of running Python and connecting to the Arduino. The following Python script reads data from the serial port that’s connected to the Arduino. The Arduino’s output is formatted so we parse this output to extract the humidity and temperature values. The values are then prepared and sent to the Graphite server. Graphite takes a simple format for its input sent to TCP port 2003 (default):

<metric_key> <value> <time>
...
garage.0.humidity 50.80 1372739553
garage.0.temp 27.10 1372739553

The example data has the key formatted in a hierarchy. The values here are 50.8% humidity and 27.10 degrees Celsius. Time is the number of seconds since epoch.

Python script for reading and sending data to Graphite

Results

The screen grab shows the daily fluctuation of humidity and temperature of my uninsulated garage in Mobile Alabama while running a 12K BTU AC unit. I prefer using Graphite compared to Google docs spreadsheet. I have higher reliability in logging the data, don’t hit the 400K max cell limit, and I can view the results updating in real-time. The Graphite setup is overkill, but I think getting exposure to the system is worth the effort. If you’d like to learn more about Graphite and how you can use it for gathering application and operation metrics check out this article from the DevOps folks at Etsy. Measure Anything, Measure Everything.