Technology / Business

Add as a preferred source on Google

Microsoft Traces Outlook Outage to Gateway Fix and Teams Routing Shift

Microsoft said the active disruption lasted nearly 24 hours; no impact was observed for more than three months after the gateway fix began rolling out.

Updated 2 min read
A commuter checks a phone outside Microsoft's campus in Redmond, Washington. / EconoTimes.ai
A commuter checks a phone outside Microsoft's campus in Redmond, Washington. / EconoTimes.ai

Microsoft said a software fix for its Microsoft 365 gateways combined with a Teams routing change and lost server capacity to leave most Exchange Online users without mailbox access during last week's outage.

In its preliminary post-incident review, Microsoft said active impact ran from 11:20 a.m. ET Nov. 12 through 11:10 a.m. ET Nov. 13. Most Exchange Online users could not reach their mailboxes through Outlook or other mail clients, and many emails were delayed. The effects varied by region, connection method and time, the company said.

Microsoft traced one contributing fault to a fix for its Microsoft 365 network gateway software, which began rolling out in July. The company observed no impact for more than three months. Under certain load conditions, competing processes inside the gateway servers exhausted resources and left machines unresponsive. Microsoft said the defect applied to those servers, not to Windows deployments generally.

The review places a Teams routing change about 15 hours ahead of the outage: Teams traffic moved from an alternate gateway to the primary one shared with Exchange Online. The change raised load and altered traffic patterns on the primary gateway. Nearly 14% of gateway capacity was already unavailable because of a separate issue the day before. Microsoft said that capacity loss alone would not have caused the outage.

As servers became unresponsive, Microsoft’s traffic controls sent requests to functioning machines. The company said the resulting load during the response came from rerouting, not an overall increase in traffic. At the worst point, more than four in five gateway servers were unresponsive. Teams, OneDrive, SharePoint and Office apps could shift away from the affected gateway, but Exchange Online could not be moved at scale. Teams chats and meetings worked for most of the disruption.

Microsoft said its tests did not reproduce the customer traffic and hardware mix that exposed the defect; newer machines with more cores were hit harder. Engineers restored the gateway code to a previously working version and restarted groups of servers. Monitoring and email recovery continued after the active impact ended. Microsoft later said it had processed the queued mail, following an earlier update on delayed email processing.

Charles Lamanna, Microsoft executive vice president of Copilot, Agents and Platform, apologized in a post on X, saying the company was “deeply sorry” for the customer impact. Microsoft says a final review will replace the preliminary account; its report gives Nov. 20 as the date for a comprehensive list of repair items.

Related articles

View all