Procedural Failings Prompted AT&T Outage, FCC Report Finds

the report said “all voice and 5G data services for all users of AT&T Mobility were unavailable as a result of the outage, affecting more than 125 million registered devices, blocking more than 92 million voice calls, and preventing more than 25,000 calls” to 911. The direct cause was “an error by an employee who misconfigured a single network element, ultimately causing the AT&T Mobility network to respond by entering Protection Mode and disconnecting all wireless devices,” the bureau said: “Adequate peer review should have prevented the network change from being approved, and, in turn, from being loaded onto the network. This peer review did not take place.” The report cited a lack of post-installation testing, inadequate lab testing and “insufficient safeguards and controls” on AT&T's part, as well as insufficient procedures for mitigating problems. It noted the company has “taken numerous steps to prevent a reoccurrence.” For instance, within two days of the outage, “AT&T implemented additional technical controls in its network,” the report found: “This included scanning the network for any network elements lacking the controls that would have prevented the outage, and promptly putting those controls in place. AT&T has engaged in ongoing forensic work and implemented additional enhancements to promote network robustness and resilience.” AT&T has "implemented changes to prevent what happened in February from occurring again," a spokesperson emailed: "We fell short of the standards that we hold ourselves to, and we regret that we failed to meet the expectations of our customers and the public safety community.”