An independent report revealed that a mistake made by Rogers staff was responsible for the 2022 outage that affected 12 million customers, causing them to lose wireless services. The outage occurred on July 8, 2022, lasting between 12 to 15 hours, with some customers unable to even dial 9-1-1 from their cellphones.
The Canadian Radio-television and Telecommunications Commission (CRTC) commissioned Xona Partners to investigate the causes of the outage.
According to the report, the core routers crashed shortly after the policy filter was removed, resulting in a disruption of various services such as mobile, home phone, Internet, business connectivity, and 9-1-1 calling.
Other contributing factors highlighted in the report included a decrease in risk assessment from high to low, remote staff’s inability to access the management network, and a lack of backup connectivity to the network operation center.
Rogers employees were unable to communicate with each other during the outage, relying on SIM cards from other mobile network providers to establish communication.
Recommendations
Prior to the completion of the report, Rogers implemented changes to prevent future outages, including strengthening network resiliency, implementing all report recommendations, and investing $20 billion in network reliability.
Rogers is collaborating with Cisco to split and build a new IP core network to prevent simultaneous loss of wireless and internet services. Internal processes were updated to enhance incident and change management processes.
The report suggested additional measures for Rogers to avoid future outages, such as testing emergency roaming, conducting root cause analysis, rigorous testing of configuration changes, and informing customers on how to reach 9-1-1 during outages.
Bogdan Diordiev and Andrew Chen contributed to this article.
Could you please rewrite this for me?
Source link
