Massive 911 Outage Caused by Single Software Glitch, FCC Determines
The failure of a single system, the result of a software glitch, led to a 911 outage in April that affected seven states and left more than 6,600 emergency calls unanswered, FCC officials said Friday. The effects were felt in nine states (see 1405200051). The FCC launched an investigation in May (http://bit.ly/1n1PvID) and commissioners received a briefing Friday at their open meeting.
Sign up for a free preview to unlock the rest of this article
Timely, relevant coverage of court proceedings and agency rulings involving tariffs, classification, valuation, origin and antidumping and countervailing duties. Each day, Trade Law Daily subscribers receive a daily headline email, in-depth PDF edition and access to all relevant documents via our trade law source document library and website.
FCC Chairman Tom Wheeler said the report was “terrifying” and pointed to “unacceptable circumstances” the FCC must address. “This is not a one-off kind of situation,” he said. Promoting the protection of public safety is “job one” for the FCC, Wheeler said. “Clearly technology is changing the way 911 is provided.” The FCC is pro-new technology “but we’re even more pro-public safety,” he said. “We have to do something about this quickly,” he said.
David Simpson, chief of the FCC Public Safety Bureau, said the incident was the result of a “preventable software error” at a center located in Englewood, Colorado, that processes 911 calls from across the U.S. Because of the software error, the signals were not relayed back to the appropriate 911 call centers, he said. “In a very real sense 911 calls got stuck in Colorado.”
The outage lasted more than six hours and potentially put more than 11 million Americans at risk, Simpson said. The problem in part is that 911 service is a “relatively small” niche market, he said. “It appears that traditional market drivers have not led to the same kind of robustness for 911 services that is enjoyed by more widely used commercially Internet-based services,” he said, adding, it is also difficult to hold parties accountable for 911 reliability, with oversight crossing multiple jurisdictional boundaries.
Simpson tied the issue to risks following an IP transition. “Single component failures in a poorly implemented tech transition 911 world impact not just one community but can impact neighboring communities as well,” he said. “The reliability of 911 communications is a unique imperative that must be proactively protected,” he said. “Calls to 911 must go through. Period.” Some failures will occur, but networks must be designed to minimize the impact, he said.
Factors related to the IP transition played a role in the outage, Simpson said. New technology has benefits, “but it can just as easily be used to lower operating costs through consolidation or hardware and software,” he said. There must be a “balance,” he said. “There can’t be an “increasing trend of larger and larger groups of consumers impacted by simple equipment failures,” he said.
“These failures are preventable,” Commissioner Jessica Rosenworcel said. “That’s what we know now because the commission staff has done extraordinary work to understand just what happened.” The FCC must get its “house in order,” she said. “While a more centralized IP-based system can mean tremendous new functionality, it also can mean new vulnerabilities,” she said. “The kind of software glitch we had here is just an early demonstration case.”
Commission staff worked with carriers, combed through data and sought advice from public safety officials, she said. “As a result, we know that this multistate outage was caused by a coding error. We know that this error stopped calls from being routed through the network to affected 911 call centers. And we know that because of a totally inadequate alarm system, this error was not detected for six hours.”
The April outage should be seen as a “wake-up call,” said Commissioner Ajit Pai. “We might not be so lucky the next time,” he said, adding that “911 call centers must remain vigilant and hold their service providers accountable.” Commissioner Mignon Clyburn also called the failure a wake-up call. “No matter how advanced our communications systems become, technologies can never replace accountability, redundancy and coordination with stakeholders,” she said. “These principles should be fundamental for any entity providing emergency communications services on behalf of a state or local government.”
“What is most troubling is that this is not an isolated incident or an act of nature. So-called ‘sunny day’ outages are on the rise,” the report argues (http://bit.ly/1tA2RkG). ”That’s because, as 911 has evolved into a system that is more technologically advanced, the interaction of new and old systems is introducing fragility into the communications system that is more important in times of dire need.”