I’ve been looking around for ideal monitoring tool for quite a long time and I can assure you that the ideal monitoring tool doesn't exist. At least not for me. That's why I've decided to create my own monitoring tool that will be as close to my ideal one as possible.

I'm not saying that the existed monitoring tools are bad or do not work. Most of them are good and do work. All I'm trying to say is that I can't really use them or enjoy using them. There are tools that are simply not doing what I want them to do. Some tools can't be installed to customer's servers. The others are too intrusive for customer's applications. On the other hand there are priceless monitoring products (meaning I can't find the price on the official web site of the vendor) and there are tools that are simply frustrating and are not fun to work with.

I'd like to share with you my createrias of the ideal monitoring tool:
  1. Setup time. How fast one can download, setup and configure the tool till it starts to provide some useful monitoring information? My own expectation is up to 1 man-day. If it takes more then a day to get some initial impression of a tool, please, throw it away and move on to the next one because most likely you are looking at marketing bullshit that will cost you a fortune.
  2. Application intrusiveness. What changes are required to be done to your application in order to enable its monitoring? Most typical changes could be like putting some jars to classpath or adding java agent to JVM, enabling JMX as well as modifying your code by annotation, by aspects or by tool-specific configuration files. You must decide how far are you ready to go with modifying your customer's application.
  3. Environment intrusiveness. What changes are required to be done to your environment and servers in order to install and run monitoring tool? Usually you will have to install some piece of software locally and remotely as well as provide connectivity like opening additional ports, adjusting firewall and proxy settings. Sometimes it's just impossible to change anything because you don't have an access to the customer's infrastructure especially in production.
  4. Capabilities. For most of you this is the most important point because it's about what monitoring information the tool is capable to provide. However if a tool does what you need but it couldn’t be installed at your servers or requires a dangerous byte-code instrumentation of your application then its capabilities worth nothing.
  5. Truthfulness. Even if tool's documentation states that the tool provides the metrics that you need you must ask yourself: how are you going to ensure that the values and plots you see on a nice AJAX screen are trustful and correct?
  6. Out-of-the-box integration. It’s nice to be able to monitor application server specific metrics or database server specific characteristics. It is valuable if a monitoring tool can handle different JMS providers. For me as a developer the next point is even more valuable.
  7. Customisation. It’s obvious that no tool can fit all sizes and satisfy all needs. That’s why it’s extremely important for tool to provide API and extension points for those developers who want to create their own plugins.
  8. Hardware monitoring. One must always remember that programs are running on top of hardware. In particular that means that application is down then whether it's native process is down or the piece of hardware the application is running on is down. That's why along with smart JVM characteristics like heap memory and object allocation count per sec the ideal monitoring tool must provide at least some primitive hardware metrics like CPU utilization, hard drive free space or network bandwidth otherwise one will never know the root cause of an application to run slower then usually.
  9. Alerts. The ideal monitoring tool should alert about something that may be interesting for the supervisor even if the supervisor is not looking at the tool's screen at this very moment. For example monitoring tool could flash a window or send email or SMS about events that are defined as critical and require immediate action to take.
  10. History. The most advanced monitoring tool must be able to store monitored metrics into embedded or pluggable database in order to allow further analysis to fix performance issues or predict trends.
In addition to more or less objective tool requirements above I’d like to point some createrias that are also important to consider while choosing monitoring tools for a long term usage:
  1. Project liveness. Is the vendor of the monitoring tool alive and still supports the product? Is there an active community of users? How many commits were done last month to the project's public VCS? Are there forums with active discussions of this monitoring tool? Can I easily contribute to the tool's project if I'd like to one day?
  2. Price. You won’t believe this but for a bunch of enterprise "world-leading" monitoring tools there is no explicit price available for observation on the official web site of the vendor.
  3. Developer-oriented. I am a developer and I'd like my monitoring tool to be easy to use by simply providing me with monitoring information that I need. So as a developer when I choose a tool I want to download it, install it and try to use it. However there are enterprise “world-leading" vendors of monitoring tools that provide meaningless marketing bullshit about the tool and no trial version to try it in action. Instead on their official web site I can only read how client oriented and customer loyal they are.
  4. Piece of art. Yes, I want my monitoring tool to be surprisingly fun to use and give me more then just a bunch of numbers and plots.
  5. Habits. Should I develop new habits in order to comfortably use a new tool? Is the way the new monitoring tool works is in line with my previous user experience as a developer?
What createrias do you consider while choosing monitoring tool for your project?

Wish you wise and pleasant monitoring,

Vladimir Krasilschik @ Kupchino Labs
Saint-Petersburg, Russia