A fix has been implemented and we are monitoring the results.
Posted Dec 13, 2019 - 20:55 UTC
Update
New hardware was installed and configured and our connectivity has been restored. We are monitoring services as they are restored.
Posted Dec 13, 2019 - 20:54 UTC
Identified
Senior datacenter networking personnel are working to replace the failed equipment and restore the service.
Posted Dec 13, 2019 - 18:06 UTC
Update
Update from our Datacenter: "We completed rebooting the router but it still is showing down. Currently one of our tier 3 technicians is working on this issue to solve it. We will provide and update when one is available."
Posted Dec 13, 2019 - 17:15 UTC
Update
A work around has been applied to restore service to builders in the vexxhost infrastructure.
Posted Dec 13, 2019 - 15:49 UTC
Update
We are continuing to investigate this issue.
Posted Dec 13, 2019 - 13:32 UTC
Investigating
There appears to be a Network Routing Issue with an upstream provider at our PDX Datacenter. This affects some core infrastructure availability. Kernel.org and PDX mirrors are affected as well as some CAF properties (wiki) and other services.
This issue is affecting builders for vexxhost infrastructure.
Posted Dec 13, 2019 - 12:25 UTC
This incident affected: Core Infrastructure - PDX.