Thursday, May 10, 2018

90-minute downtime on some services

On the evening of May 9, 2018, we were notified that the majority of our services were "down" including some client websites, e-mail, and backup services.  This turned out to be partly true, but more of a false positive as, in an unprecidented incident, all 4 of our main DNS servers were publishing incorrect "records" for a while.  Sites are still recovering, but most customers never felt the impact, due to caching.

Since it will most likely show up on our status page as a marked period of downtime, we felt it was in our best interest to report and document it here.

Apologies for any inconvenience.

No comments: