Don’t Worry – It Will Be Worse Than You Think

It is still the definitive master class case in crisis management – Tylenol’s response to the poisoning of their products in 1982.  The decisiveness, the candor, the ownership of the incident – all these elements allowed consumers to regain trust in the brand, their products and the organization.

Yet, nearly 40 years later, I still see organizations failing to utilize these concepts when it comes to cyber security incidents.  I can’t help but wonder if as CISO’s, we are too focused on preparing our organizations for the incident, and not the CRISIS that it creates.

We recently had a vendor become a victim of a ransomware attack.  They were a key part of our value chain and held customer NPI – a major supplier in our industry.  Their technical response seemed competent and timely – it was the crisis management that left us unsettled.

After several inquiries through our account management team – during and post incident – their eventual, perhaps final communication let us know that they found no evidence that NPI was accessed or exfiltrated.  Their messaging conveyed that their cybersecurity controls were OK-enough before, but getting better.  A considerable amount of space in their communciation was dedicated to letting us know that they were a victim, and that ransomware had targeted many organizations, across all industries (so, really – what could they have done?).

As I assembled our cross functional IR Team to review the incident, I asked them to remember everything that left us wanting in this supplier’s response.  Why would they leave us with impression that they weren’t taking ownership?   They seemed unaware of the lingering trust deficit their statements created.  I wanted ensure that if ever we had a similar incident, we would not make the same mistakes.  Of course, our technical response had to be effective and thorough to eliminate the threat – but we also have to regain the trust of our stakeholders.

When conducting table tops and other incident response exercises, I make sure to focus on the crisis management as well.  It is helpful to keep these concepts in the forefront of the minds of your cross functional IR teams:

  • It will be worse than we think
  • Transparency and candor are critical
  • Own what happened

It will be worse than we think

I remember the selection process when I was evaluating partners for forensics and incident response services.  We had be the big names pitch their capabilities and scale – and most were already on panel for our cyber liability insurance.  But I wanted to know in detail – for them to walk me through the first hours and days – what was going to happen if I ever needed to make “the call”.

Most gave me basically the same, totally reasonable responses about their capabilities and how we would proactively partner to ensure the response was effective.  But the winning partner, gave it to me straight, by adding the excruciating details:

  • EVERY host would need their forensics tools installed – quickly – no questions asked
  • These tools would be intrusive – critical services going to break that we would have to fix
  • The entire IR Team – not just the technical staff – should be ready for weeks of intense activity
  • There will be burnout and turnover – and the organization should be ready to support their key staff during these weeks

They basically won the contract right there.                          

Thankfully it’s never come to this – but I make sure to frame every table top we perform in this reality – to remind the team that is going to hard – not just technically to contain and eradicate the threat – but the interruption of services, the intense crucible of activity and the hard work of regaining the trust of our stakeholders.

Transparency and candor are critical

For days, our vendor who had the incident left unsure about the nature of the attack – the impact on the vendor’s systems or our customer data.  While there were early nods that NPI was not accessed, and that data integrity and source code were not impacted – there was not much underpinning these statements.

Even after services were restored – it would still be days before we had even an outline out the attack.   Eventually, and somewhat predictably, initial access was determined to be phishing and the attack confirmed as ransomware.  However, we still have no indication at what phase the attack was mitigated, or if the phishing victim host had access to critical systems or data, or if additional lateral movement occurred after initial access was achieved.

Most of the uncertainty and caginess of vendor’s statements, they attributed to the ongoing investigation.  While that it understandable, this information is critical for customers and other stakeholders to manage their own response.  Disclosing more information is always at your discretion – outside of national security or other fairly extreme cases.  Also, the answers we wanted were not likely going to impact the investigation or hamper attribution (if even possible) – but they were certainly uncomfortable.

Transparency is foundational to restoring trust in any relationship.   Senior Management needs to be ready and out front.  Public Relations and Legal teams should be prepared to facilitate the proactive outreach.  Pre-approved messaging templates should be focused on the voice and mindset of candor and ownership.

We owe it to stakeholders to share what we know honestly, so that they can protect themselves.  If bad news comes out later – it is more likely your stakeholders will view you as a dishonest broker of information – leading to the exact negative outcomes obfuscation intended to avoid (litigation, etc.)

Own what happened

Perhaps the single worst outcome was lingering impression our vendor left us with that they were just a victim.  Of course they were – but without owning that there were deficiencies in their training or tooling – how were they going to ensure that this would not occur again?

Senior Management and the Board must recognize that finding and addressing the deficiencies or vulnerabilities that were exploited is a key priority for them in the recovery phase.  It was already their accountability to ensure that due care and diligence was being applied to protect the confidentially, integrity and availability of their information assets.  If there was incident, it is plainly manifest that there was a failing in the risk analysis and mitigation efforts.  They must own this failure to fully address the short comings – a prerequisite in re-establishing a trust relationship with customers and other stakeholders.

Conclusion

There will be crushing pressure to minimize any incident during every phase of the response – but doing so will undermine your organization’s reputation in the long term.  We must prepare our organization ahead of time to withstand these impulses during our table tops and other proactive exercises.

And perhaps facing these realities – preparing for the crisis – will encourage the types of proactive investments and cross functional support needed to minimize the likelihood and impact of an incident in the first place.