NERC offers cybersecurity lessons following Ukraine grid outage

While an investigation into the world’s first cyber attack that produced a power outage continues, the Electricity Information Sharing and Analysis Center (E-ISAC), operated by NERC, issued a report that includes recommendations and lessons learned from the Dec. 23, 2015, attack on the Ukraine power grid.

One of the points in the March 18 report is that nothing about the attack in Ukraine was inherently specific to Ukrainian infrastructure, so grid operators in the United States or elsewhere should protect their systems against the tactics and attack methodology used in the incident.

The events, which included months of work and infiltration on hardware, software and communication devices of Ukrainian distribution utilities, “highlight the need to develop active cyber defenses, capable and well-exercised incident response plans, and resilient operations plans to survive a sophisticated attack and restore the system,” the E-ISAC said.

The E-ISAC “has long been at the forefront of cyber intelligence sharing,” which is a key tool for the power industry to respond to threats and vulnerabilities, NERC said in a March 21 statement. As part of that effort, the E-ISAC worked with private firm SANS to provide lessons learned about the Ukraine incident, NERC noted.

In its report, the E-ISAC said the document provides mitigation concepts for power system supervisory control and data acquisition (SCADA) protection and general learning opportunities to defend industrial control systems (ICS). The report is not to be confused with the work of a U.S. interagency team investigating the Ukraine incident, the E-ISAC said.

The Dec. 23, 2015, incident included a third party’s illegal intrusion into a utility’s computer and SCADA systems, with 30 different substations being disconnected for three hours, causing about 225,000 customers to be without power in different areas. The substations involved were seven 110-kV facilities and 23 substations of 35-kV at three different companies, but the intrusions began in March 2015, with the attacking party harvesting energy company employee credentials and using them to gain remote access to key facilities, the report noted. 

The E-ISAC report does not address who carried out the attacks, and instead focuses on the sophistication of the effort and steps companies should take to protect their systems from intrusions, since any subsequent attacks are likely to have perpetrators change tactics.

The E-ISAC said that the cyber attacks are the first publicly acknowledged incidents to result in power outages, and the motive and sophistication of the effort “is consistent with a highly structured and resourced actor” that adapted to match the defenses of the targets and used varying tactics.

The attackers used a variety of capabilities, including spear phishing emails, variants of the BlackEnergy 3 malware and the manipulation of Microsoft Office documents that contained the malware to gain a foothold into the information technology (IT) networks of the electricity companies and operating ICS through supervisory control systems, the report said.

They also targeted field devices at substations, wrote custom malicious firmware to render devices inoperable and used telephone systems to generate thousands of calls to an energy company call center to deny access to customers reporting outages, the report said.

“However, the strongest capability of the attackers was not in their choice of tools or in their expertise, but in their capability to perform long‐term reconnaissance operations required to learn the environment and execute a highly synchronized, multi-stage, multi-site attack,” the E-ISAC said.

The E-ISAC noted that media reports on the incident have focused on the BlackEnergy 3 malware or back doors to gain IT access, but those were not responsible for the outage, they were simply a component used in the attack, and “the actual cause of the outage was the manipulation of the ICS itself and the loss of control” due to the actions of the attackers. Too much focus on the malware used places grid operators in the mindset of simply waiting for specific attack components so that they can eliminate them, instead of protecting the ICS tools and devices that were used to achieve the desired effect, the report said.

Among the mitigation strategies that companies should address to protect SCADA systems and other devices, the E-ISAC report listed numerous recommendations, including passive and active defense mechanisms. Active defense mechanisms include training staff to hunt for odd communications leaving the network environment, ensuring that personnel have access to detect phishing emails, taking digital images of systems in the supervisory environment every six to 12 months, changing user and shared passwords and planning for responses to incidents among IT and operational personnel, along with other steps.

By using computer forensic tools, organizations can search for malware infections and remove the malware from infected assets, but grid defenders should be aware of the time it takes to detect an infected host as an intruder may have already breached a network and secured methods to interact and communicate with the infected network, the E-ISAC said.

Passive defense mechanisms include properly segmenting networks from each other, limiting remote connections only to personnel that need them, testing tools and technologies used in defense of network systems regularly and consideration of an event monitoring system configured specifically for high-value ICS and SCADA systems.

“Countering flexible and persistent human adversaries requires properly trained and equipped human defenders,” the report said.

The E-ISAC identified five themes for grid operators to focus on as they consider whether their systems are vulnerable to something similar to the Ukrainian incident, with the first theme involving the sequence of events and months of conduct by the attackers, which provided numerous opportunities to detect a threat and defend grid infrastructure.

The second theme is that the attacks on three different companies were coordinated and conducted within minutes of each other, and while the number of customers was not substantial, there could be significance if targets or specific loads were selected.

The third theme is that the attacks were mislabeled as solely linked specific malware, and the fourth theme is that the attackers had access to work across multiple systems, likely performing a significant amount of unobservable testing prior to the date of the outages.

The fifth theme is that information sharing is key in identifying a coordinated attack and directing appropriate response actions, the report said. While information sharing in Ukraine was not robust, in the United States and other countries with established information sharing mechanisms, the focus should be on maintaining and improving the information provided by grid asset owners and operators.

An increase in data sharing will enhance situation awareness within the power sector, lead to earlier attack detection and facilitate incident response, the E-ISAC said.

That last theme was mentioned by industry officials in February, when speakers at the National Association of Regulatory Utility Commissioners winter meetings noted that cyber threats evolve rapidly compared with NERC standards and state regulations to protect the distribution grid. FERC held a classified briefing on the Ukraine incident, and it is helping share information on reliability threats with state regulators, FERC Chairman Norman Bay said at the meetings.