...

Recent cybersecurity testing of SLAC systems and the impact on Experiment Control Systems

Alex Wallace 

SLAC will undergo a DOE cybersecurity assessment in June/July of this year. SLAC IT began preparations for this assessment starting in late March and early April. These preparations included hiring an external security testing company, Shorebreak, to conduct various cybersecurity related tests of the SLAC IT systems.

These tests included attempts to gain access to and the ability to execute processes on computing devices connected to the SLAC intranet first from the internet, then from within the SLAC network (e.g. a computer connected to a SLAC IT managed switch or router). This process includes a variety of attack methods including port scanning and attempts to exploit known vulnerabilities in IT equipment and software.

It was somewhat unsettling to learn about the upcoming assessment incidentally and the ongoing testing after it had begun. ECS has experienced control system disruption due to SLAC Cybersecurity's standard and regular port scanning activities. This disruption has manifested in the form of outages as network connected components may not handle port scanning traffic gracefully. ECS has requested on multiple occasions to be notified anytime these kinds of probing activities are taking place so we can prepare to recover affected systems and coordinate with operations. Effects can include COTS devices becoming non-communicative until power-cycled, but there are also more dramatic possibilities which are not far from the realm of possibility and may affect physical systems causing equipment and human hazards.

Our network system security and robustness is generally addressed by designing our network to be completely closed off except for a small number of explicitly identified ports. Network configuration is a complex activity and occasionally due to misconfiguration and lack of coordination we have experienced Cybersecurity induced control system outages.

In the case of the recent assessment by Shorebreak, ECS and AD EED became aware of testing after it began. After becoming aware of the ongoing assessment the Control System (Software) Working Group (CSWG) engaged with SLAC Cybersecurity to identify especially sensitive networks which would be off-limits during these tests, as well as networks would would require advance notice from Shorebreak before they began their tests so we could inform operations and prepare to recover systems. Greg White helped to ensure the SLAC Cybersecurity and relevant control system experts were able to meet and coordinate and raised awareness of these activities. McCullough, Mark was an excellent point of contact from SLAC IT and patiently helped us find a workable path towards preparation for the DOE assessment and preventing excessive disruption.

Given the present environment at SLAC with regards to work planning and control, it is somewhat surprising that this activity would proceed without a wider broadcast and approval process. Our control systems are designed and built to operate nominally within a network environment consisting of known types of traffic. Testing during installation and commissioning confirms to the extent possible that our systems are robust in the ways we designed them to be. The introduction of cybersecurity testing, which is an aspect we don't strictly consider in our designs, is risky. That risk is compounded due to lack of communication. These are lessons we must learn from. On the positive side, we now have a stronger relationship with SLAC Cybersecurity than we have had before. Also, while system security was a growing concern for the CSWG, these assessments have increased our attention and perhaps motivated us to more seriously consider the topic.

Notice regarding hardware purchasing

...