The Glaring Omission in Your Incident Response Planning

Chances are excellent that your incident response plan has a glaring omission in regards to one of the most critical aspects of success during an incident.

There has been an immense amount of time and treasure expended on what a proper incident response plan should look like.  Just throw “incident response plan” into your favorite search engine and you’ll get pages and pages of content. You’ll see all sorts of advice on how the various steps and phases of an incident response plan should play out and quite a bit of thought being put into things such as collecting contact information, identifying stakeholders and roles, inventory of tools to be used, determining secure communication methods (because you’re assuming the baddies got you email servers early and often), and the like.  Great stuff.

Does any of your plan talk about how to take care of your people during a major incident? I’m talking about those incidents that are measured in weeks or months where it’s an all hands to the pump 24/7 response measured in days or weeks of the response.  Once these incidents kick off, it’s too late for the preparation stage.  It’s show time and there is an immense amount of stress involved on all of the team whether it’s the CISO who is constantly being asked for updates by senior executives who are seeing their career dissipation lights cranked up to about a quarter million lumens or the lowest level incident responder who is cranking out digital forensic images or pouring through network logs.

An incident response plan for major incident responses isn’t fit for purpose unless it addresses how your incident responders border collies will be fed, watered, and rested. An organization should have a catering plan in place before an incident so that they can start getting a steady stream of food and drink to the people who are going to be putting in an immense number of hours all around the clock getting things under control.

If it’s a large organization (or a really nice start up in Palo Alto) chances are excellent that there is already an on-site cafeteria for employees that probably offers on-site catering services.  The incident response plan should specify how to engage those people and who the points of contact are.  You’re also going to want to talk to them before an incident to make sure that you can get food to cover a long term around the clock response.

If you don’t have anything on-site, you’re going to want to identify several external catering options and understand how to engage them on short notice for an extended response and to understand how scalable their services are since you might be feeding a very large team.  Their contact information, billing methods, and the like should be part of your incident response plan. You also need to discuss with your catering providers the menu options available before an incident. It’s important to give your people healthy food during an incident to keep them going.  Just saying you are going to order a steady stream of pizza from the take-out place down the road for weeks on end isn’t a great option.  You want to give your people some healthy options to keep them fueled up, feeling good, and ready to chase bad guys out of your network. 

You also want to make sure you are providing your people with a variety of non-caffeinated drink options in addition to the endless gallons of caffeinated sugar water or energy drinks that fuel most major incident responses.  

Keep in mind that you are going to be feeding not only your employees, but any consultants that parachute in to help you out of your bind.  There is a lot of dietary diversity these days so you’ll want to make sure you have options for people who need it due to medical, religious, or cultural reasons.  Popular options include vegetarian and gluten-free diets which works out well because you can get fantastic stuff that complies with either that everyone will enjoy.

The other thing that needs to be covered is transportation for your people.  Drowsy driving is a thing and it’s a thing you want nothing to do with during an incident.  Ride sharing services have made this much easier especially in major metropolitan areas.  The goal is to make sure you can get your people safely and efficiently back and forth between home (or the hotel rooms they are calling home during the incident) and work. Most of your people will be driving into work, but if they are too tired to drive because they ended up working a day or more in a row without sleep, it’s probably not a great idea to let them drive home and your plan should address that fact.

Which reminds me of an important point. If you are having people staying up for days on end, you’re very likely understaffed for your incident and you need to fix that quickly or you’re asking for more problems.  My general rule is that I don’t do forensics after ten hours because my chances for mistakes go up dramatically.  I’ve lost count of the amount of times that I struggled with something during a forensic exam at the end of a very long day only to solve it the issue in first fifteen minutes of being back in the office after getting some sleep.

As always, the keys to success are people, processes, and tools and your incident planning should reflect that fact.