In case you missed it, Clorox is still recovering from a cybersecurity incident. The impact of that incident has been significant enough to hurt the company’s quarterly earnings. Among many other functions that have been hit is the supply chain. The supply chain disruption resulting from the attack has led to stockout. They had to shut down manufacturing at many locations because of the system failure. As per a CNN news article (link at the end of this article):

“Still, Clorox has not been able to get its manufacturing operations back up to full speed. The company said it is fulfilling and processing orders manually. The company doesn’t expect to begin the process of returning to normal operations until next week.”

“Clorox has already resumed production at the vast majority of its manufacturing sites and expects the ramp up to full production to occur over time,” the company said. “At this time, the company cannot estimate how long it will take to resume fully normalized operations.”

Supply chains today face many risks. From a risk treatment categorization perspective, risks like the COVID-19 pandemic must be accepted. It is almost impossible to predict these Black Swan events. Then, there are other categories like mitigation and avoidance. Cybersecurity attacks should be treated within these two categories from a supply chain perspective. And even if your cybersecurity preparation or treatment strategy is in the “mitigation” category, there should be a disaster recovery plan in place, specifically in an era where manufacturing operations are heavily automated, and we talk about digital twins.

Cybersecurity preparation is not just about preventing attacks but also the ability to recover from those attacks. After COVID, every supply chain vendor claimed their solution could help build resilient supply chains. Based on specific categories of applications, I have always wondered- “How?”. But it is clear that even if a solution could help recover from a disruption, it was not a disruption like this. When the organization loses access to these systems themselves, there is no way technology can help. But the other two elements, people and processes, can make technology work.

Resiliency against cyberattack disruptions

We will focus on developing a system to build resiliency after a cyberattack has happened and crippled online systems. I will use my favorite triad of people, processes, and technology to walk through a high-level overview of such a system.

Identification of critical processes: Your ERP and other systems support hundreds of processes. Not all of them however are business critical. While every process can be impacted if you lose access to the ERP, some are critical, like the manufacturing management or order management module. As a first step, identify the critical modules within these systems. I would suggest not identifying entire systems since the approach that I am going to suggest will not be practical then.

Build a SCM system resiliency team: Every initiative starts with your people, and the success of every initiative is a result of your people. Everything else just enables these people to do their work more efficiently. Building resiliency against cybersecurity disruptions is no exception. Build a team of people, picked from business and IT, who will devote 10-20% of their time to building processes and technology for cyberattack resiliency. If you already have roles focused on supply chain resiliency, they are the natural choice. Otherwise, pick business leaders from the critical processes identified in the previous step. Pair them with their corresponding IT systems expert and a cybersecurity expert. The team must have top management support, so an executive needs to lead this task force.

Design the process: Now comes the challenging part to design the process.

There are two elements to this process.

• One is the disaster recovery aspect. When you lose access to the core systems you need to run your processes, what makeshift arrangement should be in place to keep the processes running.
• The second is, while the disaster recover setup gets triggered and is in motion, how do you recover from the attack and restore the systems.

The second aspect is what generally falls under pure IT cybersecurity teams and organizations have put a lot of effort into the recovery. We will focus on the first category here-and this is what the team created in step 1 should specialize in. How do you keep the business processes running while the systems recover from the attack.

One of the concepts of system resiliency is mirroring. For critical on-premise systems, there should be a near real-time mirroring capability. Remember that this is not running a parallel ERP or similar systems. It is mirroring the critical tables in these systems at regular intervals, on a setup that has no connectivity, whatsoever to internet. Also, because incidences like the Clorox one involved internal actors (that is what I could interpret from the news), the access to this mirror will be extremely limited.

Then, you list all the planning capabilities in the live system and document that planning logic. This will be just the critical business processes (like manufacturing scheduling and planning). Within those as well, you shortlist the ones that MUST happen to keep the operations on the floor running.

Once you have the list and associated logic, build these algorithms in an open source tool that leverages the data from the mirror. And to be candid, no matter what the marketing is, manufacturing planning and scheduling logic in most systems is not difficult to replicate, if you are looking to build an ad-hoc workaround.

Now, for your mission critical business processes, you have both the critical components. You have the most recent data, and you have the ability to generate a couple of iterations of tactical plans based on that data.

This setup will work only if you can recover fast enough and then switch the planning back to your regular systems. Why? Unless you replicate the entire ERP in your mirror, this ad-hoc planning tool cannot keep updating the data across the business processes. This means plans generated after a couple of iterations will be blind (non-optimal). And there is no value in making such an ad-hoc setup perfect. That resource should be devoted to making the capability of recovering from cyberattack system failures faster.

I indicated in a LinkedIn post approximately four years ago that cybersecurity will eventually become a significant supply chain threat. Incidences like these are just the beginning, and organizations now need supply chain-focused resiliency and recovery plans for these attacks.

References:

https://www.cnn.com/2023/09/18/business/clorox-cyberattack-production-disruption/index.html