Avoiding Clutter in Systems Management

Software Factories: Part 2 will be coming along on Monday, but for now a change of topic onto Systems Management

Out-of-the Box Systems Management Tools
There are numerous systems Management tools out there and a lot of corporates own a lot of diverse instances of these. Each one has its place in the market and plays to its particular strengths and has certain weaknesses. For example one might come already configured with everything you need to manage server harware, but may not be customisable to manage routers at all. Another tool may be completely generic and may be able to manage any device or system, but it comes preconfigured for almost nothing and you have to put the time and effort in to configure all the threshholds and metrics that you want to measure for each item in your environment.

Where is the Value?
The question is loaded of course and the answer is: “It depends”. If I am a Mail administrator and I want a system management console to tell me how my mail-system is doing then I would see little value in buying a completely generic tool and customising it to tell me how my mail system is doing. I would have to do extensive research and determine what the metrics are that I should typically be concerned about and what there respective threshholds should be in my particular setup. Over time I could tweak these until they are just right for me. Yuck. What a waste of time. I could just buy a specialised monitoring tool that tells me exactly what I need to know about mail-systems with all the threshholds already set. WRONG.

KNOW YOUR ENVIRONMENT
If you bought a specialised mail-monitoring system you would still have to customise all the threshholds and metrics for your specific environment because your servers may be scaled differently, your mailbox structures may be configured differently etc. If you make the mistake of leaving all that up to the manufacturer of the monitoring tool, then the chances are you will be ignoring most of the info the system produces and eventually say “This thing is always wrong” and untimately it will fall into disuse. Ok that’s a worst case, but my point is you need to KNOW YOUR ENVIRONMENT. There is definite value in researching what metrics are relevant for YOU and what the appropriate threshholds are in YOUR environment. So am I saying there is no place for specialised tools. Not at all. As long as you recognise the responsibility stays with YOU to manage YOUR environment, and cannot be delegated to a tool.

Where is the Value?: Part 2
If I were a departmental manager or executive, my requirements of a management system would be completely different. I would want a very general overview of the state of affairs in the organisation. In fact I would want period summary reports and would probably never look at a realtime console. So the requirement is completely different. If the underlying monitoring systems contributed their information into a centralised warehouse, I could draw a report off that and get what I needed.

Caution: Information Overload in progress!
The question is WHAT INFO DO I NEED? That is the RIGHT question (see iRobot – the movie) to start with. Too much information can be as useless as no information, except that now you have spent money on tools (and support staff for those tools) to produce that information that you never look at. Exactly the right balance of information is critical, especially in large environments.

I used to look after all the servers for a medium-sized corporate and after about six months of harassing the systems-management guys I had exactly the reports that I wanted. I sent a monthly report to the senior IT managers exhibiting the outages and incidents that occurred with a comment from myself on each. I also had a weekly report summarising the top 10 major potential and real issues across all servers. Each week I tried to deal with the issues on the top-most-unhealthy server. I also had a daily report on the configuration and health of each server, which I archived and kept for 3 months, which I only looked at if I needed to research something later (reading that report daily would have been overload).

It shouldn’t be easy
My parting comment is that you need to get to grips with your environment and do the investigation and research into what it is that YOU need to manage in YOUR enviroment. It is extremely unlikely that someone can do that for you. You need to make the time to do it. It will save you (I say that deliberately – it may save YOU) in the long run.

Technorati Tags: Systems Management, Operations Management