Tracking Stats With Collectd

Collectd is a daemon which collects performance statistics from a wide variety of sources and stores them in several ways. You can configure every aspect of it in its configuration file. Getting started with collectd is easy with Homebrew:

brew install collectd

Once collectd is installed, you can configure it with plugins for reading and writing data. If your Homebrew installation is located in /usr/local, you can find the configuration file at /usr/local/etc/collectd.conf. To load a plugin, first find the plugin in the “LoadPlugin” section. This will look something like the following:1

##############################################################################
# LoadPlugin section                                                         #
#----------------------------------------------------------------------------#
# Lines beginning with a single `#' belong to plugins which have been built  #
# but are disabled by default.                                               #
#                                                                            #
# Lines begnning with `##' belong to plugins which have not been built due   #
# to missing dependencies or because they have been deactivated explicitly.  #
##############################################################################

##LoadPlugin amqp
#LoadPlugin apache
#LoadPlugin apcups
#LoadPlugin apple_sensors
#LoadPlugin ascent
#LoadPlugin battery
#LoadPlugin bind
##LoadPlugin conntrack
#LoadPlugin contextswitch
LoadPlugin cpu
##LoadPlugin cpufreq
#LoadPlugin csv
#LoadPlugin curl
##LoadPlugin curl_json
#LoadPlugin curl_xml
##LoadPlugin dbi
#LoadPlugin df
LoadPlugin disk
#LoadPlugin dns
#LoadPlugin email
##LoadPlugin entropy
#LoadPlugin exec
#LoadPlugin filecount
##LoadPlugin fscache
##LoadPlugin gmond
#LoadPlugin hddtemp
LoadPlugin interface
##LoadPlugin iptables
##LoadPlugin ipmi
##LoadPlugin ipvs
##LoadPlugin irq
##LoadPlugin java
##LoadPlugin libvirt
LoadPlugin load
##LoadPlugin lpar
##LoadPlugin madwifi
#LoadPlugin mbmon
##LoadPlugin memcachec
#LoadPlugin memcached
LoadPlugin memory
##LoadPlugin modbus
#LoadPlugin multimeter
##LoadPlugin mysql
##LoadPlugin netapp
##LoadPlugin netlink
LoadPlugin network
##LoadPlugin nfs
#LoadPlugin nginx
##LoadPlugin notify_desktop
##LoadPlugin notify_email
#LoadPlugin ntpd
##LoadPlugin nut
#LoadPlugin olsrd
##LoadPlugin onewire
#LoadPlugin openvpn
##LoadPlugin oracle
#LoadPlugin perl
##LoadPlugin pinba
##LoadPlugin ping
#LoadPlugin postgresql
#LoadPlugin powerdns
LoadPlugin processes
##LoadPlugin protocols
##LoadPlugin python
##LoadPlugin redis
##LoadPlugin routeros
##LoadPlugin rrdcached
##LoadPlugin rrdtool
##LoadPlugin sensors
##LoadPlugin serial
#LoadPlugin snmp
#LoadPlugin swap
#LoadPlugin table
#LoadPlugin tail
##LoadPlugin tape
#LoadPlugin tcpconns
#LoadPlugin teamspeak2
#LoadPlugin ted
##LoadPlugin thermal
##LoadPlugin tokyotyrant
#LoadPlugin unixsock
LoadPlugin uptime
#LoadPlugin users
#LoadPlugin uuid
##LoadPlugin varnish
##LoadPlugin vmem
##LoadPlugin vserver
##LoadPlugin wireless
#LoadPlugin write_http
##LoadPlugin write_redis
##LoadPlugin xmms
##LoadPlugin zfs_arc

You should see a “LoadPlugin x” where x is substituted with the plugin name. This may be prefixed by 0–2 hashes. If it has no hashes in front of it, it is loaded. If it has one hash in front of it, it is not loaded but it is supported by your system. If it has two hashes at the start, it is not supported by your system: you are probably missing a required library.

Following these rules, you can load a plugin that has one hash prepended to it by removing this hash. You can also unload a plugin by prefixing it with a hash.

Next, configure the plugin in the plugin configuration section where plugins are configured with syntax like this:

<Plugin apache>
  <Instance "local">
    URL "http://localhost/status?auto"
    User "www-user"
    Password "secret"
    # CACert "/etc/ssl/ca.crt"
  </Instance>
</Plugin>

You can look up the configuration for each plugin on the collectd website.

Once you’ve got collectd tracking statistics, it’s time to output these statistics. I view them via a web interface called Visage. It’s a gem that can be installed by running gem install visage-app. The installation instructions say Visage can be started with visage-app start but that didn’t work for me. I used sudo $(dirname $(dirname $(gem which visage-app)))/bin/visage-app start. You should be able to run this on start up with launchctl.

When Visage is running, you can find it at 127.0.0.1:9292 (replacing “127.0.0.1” with your hostname if necessary). Start by creating a profile, where you select data you want graphed. Then hit enter and you can see the graph. Deletion of profiles is not currently supported but you can remove them manually from the YAML file they are stored in ({Ruby gems folder}/gems/visage-app-1.0.0/lib/visage-app/config/profiles.yaml). I use rbenv and my file is located at /Users/Henry/.rbenv/versions/1.9.3-p0/lib/ruby/gems/1.9.1/gems/visage-app-1.0.0/lib/visage-app/config/profiles.yaml.

Using the network plugin, it’s possible to have one Visage server for all your computers. I use this approach for my computers, they all report back to a central server. The data can then be grouped together in Visage.

I’ve been using Visage for several months without problems after switching from my own home-baked solution for tracking stats. For more info, see:

  1. Yours will probably have fewer unsupported plugins, I’m running this on my MacBook not long after a clean install.