Cio Made A Dangerous Mistake And Ordered His Security Team To Implement It

Who, Me? Welcome to another instalment of Who, Me? It’s The Register‘s reader-contributed column that shares your missives about massive mistakes, and how you managed to move on after them.

This week, meet a reader who asked to be Regomized as “FireBug,” a name that makes sense because the story he sent concerns a firewall he worked on during his time as part of a small team that managed a global company’s security and VPN infrastructure.

“I had just passed my CCNA and CCNE certifications when I received a request to make a major update to the firewall rulebase,” FireBug told Who, Me?

As a sensible fellow, FireBug had a three-stage process for such requests.

First, he would read firewall rules to check for obvious errors. Next, he would deploy them in an isolated test environment. If that worked, he would deploy into production.

On this job, he had to insert an extra step.

“A new CIO had arrived at the company,” FireBug recalled. “He was a nice guy, but more tactical than strategic, with a very hands-on approach and keen interest in everything related to security.”

The new boss wanted to review the changed firewall policies himself.

FireBug had already performed his first step – reading the rules to check for errors – and found a big problem that he felt would see the firewall isolate itself from the internet, taking the business offline. The error was so dangerous that FireBug checked it with a colleague, who agreed with his findings.

But when the CIO reviewed FireBug’s work, he found no flaws and ordered the rules be implemented.

He even put it in writing, leaving FireBug no alternative but to implement the flawed policies.

“Things went exactly as expected – the firewalls in the offices around Europe all received the update on schedule and went dark, resulting in immediate declaration of a major incident,” Firebug wrote.

The company quickly implemented its disaster recovery plan and brought plenty of its infrastructure back online at decent speed.

But some of its kit was so broken that the only way to fix it was to get hands-on.

“I had a really interesting summer travelling around Europe to places I had never considered visiting,” FireBug told Who, Me?

The CIO survived the incident.

“He had a badly bruised ego and upper management insisted that from now on [he] leave operations to us techies and focus on management.”

Has your boss ever ordered you to break things? If so, click here to send email to Who, Me? – The Register would love to share your story. ®

Original Source