T-Mobile's nationwide outage is now over: Here's what went wrong (Update)
T-Mobile has gotten to the bottom of Monday's outage, which impacted customers of other networks, too.
Editors' Note: This article was originally published on June 15 during T-Mobile's network outage. We've posted an update now that the carrier was outlined the cause of the problem.
Did you have trouble getting a cell phone connection Monday (June 15)? You weren't alone. Cellphone users flocked to social media and outage tracking sites to report problems with placing phone calls.
But the problem, centered around T-Mobile' cellular network, was fixed as of 1 a.m. ET on Tuesday, with the T-Mobile reporting that voice and text service had been restored.
- Best and worst phone carriers
- Who's got the best unlimited data plan?
- Just In: Samsung Galaxy Note 20 release date just leaked
So what happened? Based on Monday reports at DownDetector.com, the troubles started around noon ET on June 15, with the majority of users complaining of calls not going through. The vast majority of the complaints seem to involve T-Mobile subscribers, though DownDirector shows customers of other carriers noting outages as well.
It became pretty clear that the outage impacted just one carrier, with ripple effects extending to customers of other services placing calls to the impacted carrier.
Tom's Guide staffers were able to place calls using Verizon and AT&T phones to other customers on those networks. Our T-Mobile phone could dial out, but calls placed to that phone failed.
T-Mobile president of technology Neville Ray took to Twitter Monday to confirm that there was "a voice and data issue that has been affecting customers around the country." Ray said that T-Mobile engineers were working to restore service.
Sign up to get the BEST of Tom's Guide direct to your inbox.
Get instant access to breaking news, the hottest reviews, great deals and helpful tips.
Our engineers are working to resolve a voice and data issue that has been affecting customers around the country. We’re sorry for the inconvenience and hope to have this fixed shortly.June 15, 2020
During the outage, we contacted AT&T, and a spokesperson told us that the carrier's network "is operating normally." AT&T also took to Twitter to say problems may be connected to its customers trying to reach other networks.
Our network is operating normally, but it’s possible some customers are unable to reach people on other carriers’ networks. https://t.co/qPmjZnqEFAJune 15, 2020
Likewise, a Verizon spokesperson told us all is well with that carrier's network, too. "Verizon's network is performing well," the spokesperson said. "We're aware that another carrier is having network issues. Calls to and from that carrier may receive an error message."
According to a T-Mobile blog post Tuesday evening, T-Mobile engineers pinpointed a leased fiber circuit failure from a third party provider in the southeast US as the source of the problem. "This is something that happens on every mobile network, so we’ve worked with our vendors to build redundancy and resiliency to make sure that these types of circuit failures don’t affect customers," T-Mobile's Ray wrote. "This redundancy failed us and resulted in an overload situation that was then compounded by other factors."
The resulting overload in the southeast created capacity issues across the IP multimedia subsystem network that supports voice calls over LTE, Ray said. As a result, voice calls failed, though data connections and non-VoLTE services like FaceTime, WhatsApp and others continued to work.
"We have worked with our IMS (IP Multimedia Subsystem) and IP vendors to add permanent additional safeguards to prevent this from happening again and we’re continuing to work on determining the cause of the initial overload failure," said Ray, who also apologized for the outage.
Philip Michaels is a Managing Editor at Tom's Guide. He's been covering personal technology since 1999 and was in the building when Steve Jobs showed off the iPhone for the first time. He's been evaluating smartphones since that first iPhone debuted in 2007, and he's been following phone carriers and smartphone plans since 2015. He has strong opinions about Apple, the Oakland Athletics, old movies and proper butchery techniques. Follow him at @PhilipMichaels.