Control Systems Can’t Hide from Hackers, Don’t Ruffle Researchers

Control systems used to run on closed, proprietary platforms: “Security was never really taken into consideration,” Ramirez said. But the introduction of commercial off-the-shelf products in critical infrastructure, using friendly Web interfaces, also made such systems more vulnerable to hacking, he said. The ToorCon hacking conference in 2005, through its “RootWars” attack simulation competition, showed how to take down SCADA systems at a nuclear power plant, he added. “The IT industry has gone through this evolution” on cybersecurity, but “a lot of [infrastructure] vendors are in denial. Some are very angry.”

Relationships between vendors and security researchers can be tense, Ramirez said. Researchers get “antsy” when vendors are slow to, or don’t, respond to warnings of vulnerabilities: The protocol is “almost like a gentlemen’s agreement… When someone breaks it, they're going to pay for it,” he said. Researchers may get fed up and commit a “grayhack” -- releasing the vulnerability to the public to shame the vendor into taking them seriously. Though researchers rarely release all the “puzzle pieces” of the exploit, it’s possible for bad actors to reverse-engineer the hack from limited information and make the code “wormable,” he said: “You are potentially making [control systems operators] targets.” The perspective of operators must not be ignored; they may see a flaw as a simple bug that can be remedied with a “silent fix,” releasing a patch with no explanation -- as opposed to a vulnerability requiring “massive resources” and broad notification that could unnecessarily scare the public, Ramirez said.

That tension is where the U.S. Computer Emergency Readiness Team (US-CERT) at DHS comes in, screening out baseless reports of vulnerabilities and noting trends where industry sees isolated events, Ramirez said: “What we're trying to do is build trust with vendors.” Vendors usually take researchers more seriously when connected through US- CERT and given a chance to simulate the reported flaw within the vendors’ own environment, he added. “All we're really doing is validating… an already known issue.”

Microsoft threw up a wall of opposition when DHS said it would warn the public of the Windows hole, calling it a wormable threat that could significantly harm Windows- dominant critical infrastructure through remote access, Ramirez said. The company said it didn’t have a patch ready and that notification would only increase the damage from exploitation. Through discussions over a month, Microsoft acquiesced to a “prenotification” disclosure to vendors, owners, operators and Information Sharing & Analysis Center branches.

Within 5 min. of the warning’s going out to the public from DHS after Microsoft finished a patch, Ramirez said he got several phone calls and e-mails asking “What the hell did you do?” His response: “When we find information that we deem important enough, we will push it out.” At a later session, SCADA sector officials “vented for about an hour and a half” about getting “picked on” by DHS, but the agency is reconciling with them, Ramirez said.

Vendors may underestimate how widely deployed -- and unaccountable -- their products are. Some customers may not keep a paid subscription to security updates for control system products they purchased long ago. A vendor scolded Ramirez at a Cal. meeting for telling the audience that older versions of products may be on the market and never get needed updates. A few minutes later, the vendor apologized, saying he found old, unpatched versions of his products on eBay, sold from the former Yugoslavia, Ramirez related. Vendors should make it easy for researchers to contact the relevant personnel to handle vulnerability reports, Ramirez said; an audience member joked that researchers should look at companies’ 10-K filings and harass executives.