For the thousands of IT staff that are on-call, this is a familiar, and very much unwelcome, experience. Over the years, systems support staff have seen their sleep interrupted by beepers, Blackberry vibrations, and cell phone calls. I guess this beats having the home phone ring and wake up the spouse and kids. It is hard enough getting yourself out of dreamland and back to the cold back-to-work mentality in the middle of the night.
I remember the first time I received a beeper to be on-call in the early 90s. I felt so important! Whenever I was at a party I made sure to wear my beeper proudly on my belt, never tucked away in my pocket. And when it went off, it was with great pride I would proclaim just loud enough for all to hear, Excuse me but I need to call into the computer center.
|Smart IT Columns|
Understanding Your 'Idiot' Manager
Is Everyone Replaceable?
Managing Vendor Demonstrations
Blogging Your Way Up the Career Ladder
That got old very, very fast.
When the beeper started going off in the middle of dinner, during a crucial point in my favorite TV show (this was pre DVR), and repeatedly in the middle of the night, I started to think being on-call wasnt a very glamorous responsibility. As a matter of fact, it was the pits. And sometimes after working into the wee hours of the morning to solve a problem, I had to be back at work very early writing more code while feeling groggy and irritable. I began to wonder how the heck we could do more to reduce the number of calls or at least make the life of on-call staff a bit less stressful.
It should be quite obvious that better coding standards and techniques will reduce the number of bugs. And you dont have to work in a CMM or ISO9001 environment to have processes in place. However, in many IT shops the staffing levels still havent recovered from the post-bubble staff reductions and it is human nature to look for time saving short cuts when we are overworked. It is also true in small to mid-size businesses that there isnt a dedicated support staff and that every team member must take a turn at support or be in the escalation hierarchy.
So I get that everyone is stretched. However, to reduce your late night calls, you must not only put standards and processes into place, but put in a mechanism to enforce them. Im a big fan of peer-to-peer, standards-based, code and unit test reviews without a manager in the room. Teams can do a very good job of policing themselves, and with more vocal participation. Once code passes this review, then passes quality assurance and integration testing, it can be blessed for a promotion to your production environment.
A developer should never be able to promote their code into production without following this process except while on call. If they have to fix something to get the system running again, then consider it a temporary patch that must then go through the same review and testing process, preferably the next day.
At least today remote access is fast and easy. Back in the Stone Age, dialing into the mainframe on a 2400-baud line was very time consuming. Make sure your team has the tools in place at home to remotely diagnose a problem, such as Citrix Presentation Server and a high speed Internet connection. Dont ask them to rely on their home computers for support. You dont want their teenager to download a virus on a computer they need for support. Either assign them a laptop or have one that is shared by the support team.
Next page: More Solutions for On-Call Help