How to install and use RRDTool to speed up MRTG monitoring

Tobi Oetiker's MRTG is one of my most-used pieces of software for graphing system status and CPU/network activity. (For more on this, refer to my previous tutorial on installing and configuring MRTG.) However, MRTG's default configuration has one minor flaw -- with a large number of hosts or devices to monitor, the system running MRTG can have a CPU spike every five minutes or become overloaded from continuously generating graphs. One solution for this problem is to integrate another of Oetiker's software products, RRDTool. RRD stands for Round Robin Database, and the tool is designed to store data such as output from MRTG and provide an easy way to generate graphs on the fly from the RRD.

This is a basic tutorial on "plugging in" RRDTool support to MRTG and using a third-party Common Gateway Interface script to create graphs from the RRD database files. This can be done without loss of historical data -- old MRTG log files are converted to the new RRD format.

As in previous articles, my examples will be provided using a Solaris 10 x86 Update 7 VMware virtual machine running in 32-bit mode, which has an MRTG installation monitoring its own primary network interface. RRDTool can be used on just about any system that will run MRTG -- Solaris, other Unix distributions, Mac OS X, most types of Linux and Microsoft Windows.

MRTG/RRDTool on Solaris 10: Installation prerequisites
This tutorial will assume that you've installed the prerequisites listed in my previous article on setting up MRTG (the GCC compiler, Apache 2 and required packages, system search path and CRLE or LD_LIBRARY_PATH settings, etc.)

I recommend installing a current version of Perl under /usr/local, separate from the Sun-provided Perl5 install. This is because Sun's Perl is built with the Sun compiler suite, and trying to build additional modules for that Perl binary will attempt to use Sun's compiler flags with GCC. This will cause intense pain and suffering in addition to not working very well. An updated Perl package can be downloaded from SunFreeware or built from source; compiling and installing it is well documented and not covered here.

When building RRDTool from source, I prefer to use the older 1.2 branch, which is still being maintained. Massive changes in the graphics routines were made in version 1.3 of RRDTool, which make it difficult to properly compile on some non-Linux/GNOME platforms.

Building and installing RRDTool
This tutorial uses version 1.2.30 of RRDTool, which is current as of January 2009 and available for download from http://oss.oetiker.ch/rrdtool/pub/. Download the source archive and extract it in your desired build location; I prefer to use /usr/local/src. Once the source archive is extracted, change to its directory and configure the build with this command:

./configure --disable-tcl --disable-ruby --disable-python

Note: If you encounter problems later in this tutorial with missing shared libraries, use crle to add /usr/local/rrdtool/lib to your system library search path. This will set up the build environment so that RRDTool only uses the built-in Perl bindings and not those for TCL, Ruby or Python. Once the configure process is completed (you will see a "Config is DONE!" message), follow the directions displayed and run a "make" to build RRDTool and then "make install" to install everything in /usr/local/rrdtool-1.2.30.

I usually make a symlink of /usr/local/rrdtool pointing to whatever the currently installed version is; that way, when I upgrade RRDTool, I don't have anything pointing to an old directory.

Converting MRTG to use RRD data storage
Converting an existing MRTG installation to use RRDTool is easy. Once RRDTool is installed, only three lines have to be modified in the mrtg.cfg file. At the top of the file, add these lines: LogFormat: rrdtool PathAdd: /usr/local/rrdtool/bin/ LibAdd: /usr/local/rrdtool/lib/ LibAdd: /usr/local/rrdtool/lib/perl/

The next time MRTG runs, your log file(s) will be converted into .rrd files and PNG-format graphs will no longer be created or updated. Runtime for MRTG should now be about 20% of what it was previously. The old .log files can be safely deleted or moved.

Installing the graph generator
In order to view your device's graphs, we now have to use a third-party front end to generate them on the fly. Oetiker lists a number of them here, and my favorite is mrtg-rrd by Jan Kasprzak.

Download and extract the mrtg-rrd-0.7.tar.gz file to a location of your choice, uncompress it and copy the mrtg-rrd.cgi file to the directory containing your MRTG data and configuration files. Change file permissions to make it executable, then edit it, making sure the path to the Perl executable is correct. Just before the line that reads "use RRDs;", add this line to make sure that the script can find the RRDTool libraries:

use lib qw(/usr/local/rrdtool/lib/perl);

One other change to the CGI is needed. Just under the line that reads "EDIT THIS to reflect all your MRTG config files," change the path listed to reflect the proper path to your mrtg.cfg file. In this case, it would be /usr/local/apache2/htdocs/mrtg/mrtg.cfg.

In order to properly run this CGI, a couple of minor configuration changes need to be made to the Apache Web server's configuration and file permissions. If you are running the Apache 2.2.11 package from SunFreeware as specified in my last tutorial, the changes are as follows:

1. Uncomment the line that reads "AddHandler cgi-script .cgi".
2. In the Options line for the directory /usr/local/apache2/htdocs, add "ExecCGI" to the end.
3. Change the owner of the directory containing the MRTG config and data files to the user account that runs the Web server, or otherwise give that user account write access to the directory (so it can write graph files).
4. Restart the Apache Web server (/usr/local/apache2/bin/apachectl graceful)

If everything goes well, you should be able to bring up the URL of the CGI script and have it display an index page similar to that previously produced by "indexmaker" from the MRTG package:

Click to enlarge.

You can click on a graph on the index page to drill down to that interface's historical data:

Click to enlarge.

As you can see, the graphs created by RRDTool + mrtg-rrd are basically identical to those produced by MRTG alone, except for one important fact - these are generated on the fly when someone loads the CGI with a Web browser. This leads to more efficient use of system resources and lets a single system monitor many more hosts or interfaces than it would normally be capable of.

Further enhancements might be to configure the Apache Web server to use mrtg-rrd.cgi as the DirectoryIndex for your MRTG directory; I personally rename it to index.cgi and configure things appropriately.

Further reading on RRDTool
The RRDTool documentation and tutorials are great resources and explain how to use RRDTool with just about anything you'd want to plug it into. Use of RRDTool is not limited to MRTG integration.

ABOUT THE AUTHOR: Bill Bradford is the creator and maintainer of SunHELP and lives in Houston, Texas, with his wife Amy.

What did you think of this feature? Write to SearchDataCenter.com's Matt Stansberry about your data center concerns at [email protected].