Monitoring – Updated and Revisited
Tuesday, September 22, 2009 at 10:18AM I’ve written several times on this blog about various monitoring tools and services. But, in light of a recent project I’ve been working on for the last few months I have some updates.
The Overall Monitoring Architecture
There are areas of monitoring that I usually like to pay close attention to with a live web application.
- Process Monitoring - This makes sure things are running and stay running within certain tolerances. Examples are God, Monit, SMF. Your choice will depending on your operating system and preferences with scripting languages.
- Resource Monitoring - This is fine grained CPU, Memory, Disk Space, Disk IO, Networking, application server threads, and much more. Examples are Nagios, Ganglia, and Munin. Choosing correctly depends on your specific situation. There is a worth newcomer on the block called Reconnoiter that also looks very promising.
- UpTime Monitoring - This is the only monitor people usually do if they do any at all. This should be a disinterested 3rd party to provide accountability and what I call a 3rd party eye in the sky should any dispute about uptime arise. I like pingdom and there are even free services as well. I’ve also been using CloudKick in some situations for this purpose as well.
Those three above are from a post I wrote some time ago. Today, I’m adding a 4th item to that list because it has finally become easy enough and reasonably affordable to add now that there is an affordable choice:
4. Synthetic Transaction Monitors – These actually perform tests of processes a user might go through in your application and report back any anomalies if they occur along with an error report, screen shot, and other data as appropriate. I’ve been using a tool called BrowserMob and Selenium IDE for this. You create scripts w/ Selenium, upload them to browswermob and then setup a monitor script. That’s a simplified overview of course but it’s really quite effective and relatively affordable compared to historical solutions for synthetic transaction monitoring. Historically it was prohibitively expensive to do synthetic transaction monitors.
The Monitoring Tools I use
What follows are some of my personal current favorites to meet the above goals.
Munin > http://munin.projects.linpro.no/
One of the things on my list for a while to get done is enable munin across your systems. I use it a lot successfully. You can see a demo here:
Monit > http://mmonit.com/monit/
Pingdom > http://www.pingdom.com
Things I’m testing and have high hopes for are Reconnoiter, BrowserMob monitoring, CloudKick
monitoring in cloud computing, scale, web 
Email Article to Friend
Reader Comments (2)
Munin and Nagios can be used jointly ...
In can recommend also: FAN: http://fannagioscd.sourceforge.net/drupal/
Another important level of monitoring is at the application level: what's the average page load of the homepage? How long does it take to run a background archiving task? Basically, the metrics that end users notice.
I'm the co-founder of Scout, a hosted server monitoring service that makes it easy to track appication-level metrics and resources. If you're looking for an easier setup than Nagios or Munin, you might want to check out Scout.