Incidents

DESCRIPTION: UTOPIA maintenance BR005 power testing

MAINTENANCE START TIME: 6/16/2023 12:01 AM

MAINTENANCE STOP TIME: 6/16/2023 6:00 AM

SCOPE OF IMPACT: We don’t expect to lose any customers however with this type of maintenance there is always risk. Potentially customers could see a brief outage while switches reboot.

DESCRIPTION: UTOPIA maintenance power maintenance in LA009 core site.

MAINTENANCE START TIME: 6/9/2023 12:01 AM

MAINTENANCE STOP TIME: 6/9/2023 6:00 AM

SCOPE OF IMPACT: All of Davis county may experience a brief outage during this maintenance window.

DESCRIPTION: UTOPIA maintenance LI005 power upgrade/testing

MAINTENANCE START TIME: 6/2/2023 12:01AM

MAINTENANCE STOP TIME: 6/2/2023 6:00 AM

SCOPE OF IMPACT: All customers will experience a brief outage during maintenance.

DESCRIPTION: UTOPIA maintenance power upgrade/testing in OR018.

MAINTENANCE START TIME: 5/26/2023 12:01 AM

MAINTENANCE STOP TIME: 5/26/2023 6:00 AM

SCOPE OF IMPACT: All customers will experience a brief outage while maintenance is performed.

DESCRIPTION: UTOPIA Maintenance BR001 switch retirement.

MAINTENANCE START TIME: 5/25/2023 9:00 AM

MAINTENANCE STOP TIME: 5/25/2023 1:00 PM

SCOPE OF IMPACT: Residential customers will be moved during the day one at a time. Each customer will experience a 1-2 minute outage while they are moved to a updated switch.

UTOPIA Network Outage Notification for 5/23/2023 - Footprint(s) MR001 DESCRIPTION: UTOPIA has been alerted to an outage in the MR001 footprint(s) in Morgan, UT. Field Services are in transit to the site. There are 1139 residential customers affected.

UTOPIA Network Outage Resolution - 5/23/23 - Footprint(s) MR001 in Morgan, UT. RESOLVED: The outage has been resolved. All services have been restored. fiber cutover completed RESOLUTION TIME: 10:00 PM

XMission is experiencing a DNS issue and actively investigating. This problem can cause websites to fail to load and create the appearance of an Internet outage. Most XMission services are impacted. No ETA but we're all hands on deck and expect a fairly quick resolution.

Issue resolved 11:45 AM

UPDATE 1:16 PM

Incident reopened as some resolution is again unstable. Mitigations are being implemented.

UPDATE 3:50 PM

The issue is a massive DDOS to our DNS infrastructure which has been particularly difficult to fully quell. Customers should be able to use alternative DNS servers (ie 1.1.1.1 or 8.8.8.8) in their network configs as a work around.

UPDATE 6:15 PM MDT

XMission believes this issue is resolved.

UPDATE: This outage is due to a fiber cut. Field Services are working on resolving the issue. ETR: 3:00AM

A single server in our infrastructure required a full service restart. All systems are operational.