After a review of yesterdays DNS issue, today we've improved our resiliency by adding a Secondary DNS provider, so requests can now failover in the event our provider is having issues.

Avatar for Robert Rawlins
Robert Rawlins
Resolved

Hurrah. We finally have the green light from our provider to confirm that everything on their end is now back to normal.

Avatar for Robert Rawlins
Robert Rawlins

Our provider is now happy they have a fix in place, and that traffic appears to be returning to normal.

They are now monitoring the last few affected regions

Hopefully a final update soon.

Avatar for Robert Rawlins
Robert Rawlins

Still no confirmation on a complete resolution from our DNS provider, however, it's been nearly 2 hours since our monitoring last picked up any real-world impact on us.

I'm happy that we're not substantially impacted, but I'll leave this incident open until we get confirmation.

Avatar for Robert Rawlins
Robert Rawlins
Recovering

Our monitoring hasn't seen any further DNS issues for quite some time, however, our provider still believes they are having problems, so we're leaving this open until we've had confirmation that things are well and truly back to normal.

Avatar for Robert Rawlins
Robert Rawlins

Our DNS provider has assured us they are "all hands on deck" working to find solve the problem. We'll have another update within the next 30 minutes or so.

Avatar for Robert Rawlins
Robert Rawlins
Investigating

Oh-oh, looks like we may have a few intermittent issues folks. A few requests from certain regions to view Status Pages, Management UI and the Rest API look to be failing.

The root of the issue is with our DNS provider, who are aware of the problem and working to fix it.

I'll be back when I know more.

Avatar for Robert Rawlins
Robert Rawlins
Began at:

Affected components
  • Status Pages
  • Management UI
  • REST API