Sunday nights screw-up naming the Best Picture at the Oscars offers a lesson in defensive thinking, if not exactly in Defensive Computing.
The two on-site PricewaterhouseCoopers representatives had a simple job, giving out envelopes. The task is so simple, they could not imagine anything going wrong.
We know this because of interviews with each of the PwC representatives, Brian Cullinan and Martha Ruiz, before the Academy Awards, where they were asked about the plan should something go wrong. They fumbled. Something going wrong was inconceivable.
We can all forgive Warren Beatty, no one in that position would be expecting the wrong envelope, after all it had never happened before. Plus, thinking on your feet in front of millions of people has got to be hard.
But, had they been prepared for mistakes, the PricewaterhouseCoopers representatives would have been paying attention in the wings.The replay shows that the first thing Beatty did when he saw the card was to look in the envelope for another card. You also see him looking off-stage just a few seconds later. That should have gotten the attention of the on-site auditors. But, why pay attention if mistakes so unlikely?
The accountants are also supposed to memorize the award winners. So, as soon as they both heard the wrong movie announced, they should have run around as if their hair was on fire. You would think they would run on stage to immediately correct the mistake. But no, they were just as much a deer in the headlights as Warren Beatty.
The producers of La La Land were making acceptance speeches for more than two minutes. Chaos and confusion were everywhere.
This was clearly a planning failure. What could possibly go wrong handing out the right envelope for the right award? Especially the last award.
To bring this back to computing, look around your office and/or home. What if all the computing devices there disappeared; maybe stolen, maybe destroyed in a fire, maybe rendered useless by sun spots. What would you do? Planning for that is true Defensive Computing.
I don't claim to do a great job of this personally, but I have thought about it and planned for it. My personal files that change most often are sent off-site daily by a Network Attached Storage (NAS) device. Its an automated process at the end of every day.
My most important files are sent off-site once a month, manually. They are all in one folder, and I make a compressed encrypted copy of that folder into a single file. When I remember, I make two off-site backups of these files, to locations that are a couple thousand miles apart. When I forget, I make only one off-site copy. The reminder to make off-site backups is a recurring event in the calendar on my phone.
On a less drastic note, can your local (on-site) backups resist ransomware? That is, can you recover from an attack that corrupts both the original files and network accessible backups?
The easy response is to keep some backups off-line, except, of course, when they are being created. This is what I do. In addition to backups on a NAS device, I also backup my most important files to a dedicated USB flash drive that is off-line when not making backups.
A higher end approach would store backups on a device that offers file system level snapshots (i.e. BtrFS or ZFS). This would let all the files be quickly restored to the snapshot just before the ransomware infection.
My email provider has also failed to plan for the worst. They came close, they maintain a website with status information, that has a different name from the main website. But both domains are registered with the same registrar and both live in the same data center.
Just a couple days ago, Amazon made the same mistake. Amazon web services was not fully operational for 11 hours. But the mistake that interested me was that they could not put out a status update because their status system depended on the stuff that was failing. They were forced to use Twitter to provide updates on the situation.
Defensive Computing in a nutshell: expect the worst and plan for it.
UPDATE: March 16, 2017. Clarified that on-site backups need to be ransomware resistant.
- - - - - -
Now that Computerworld, and all of parent company IDG's websites, have eliminated user comments, you can get in touch with me privately by email at my full name at Gmail. Public comments can be directed to me on twitter at @defensivecomput