Mid-Day Downtime explained

This post serves to explain two downtime scenarios that occurred in distinctly different regions of our infrastructure. 



The first that occurred this morning concerns continued intermittent downtime in the Fibernetics datacenter in Canada (although the issue is more related to the tenant IN the data center where we have our equipment housed, not necessarily the datacenter itself).  We shall continue to monitor these situations and plan to MOVE the respective services to a new datacenter or hardware if the problem continues.  Customers should know that these specific servers never did contain mission-critical customer data, (though our main website was on one the servers at one point, and has since been moved).  In most cases this hardware and related services are used as a testbed prior to deploying production systems.



The second downtime scenario today revolved around hardware in a New York datacenter operated by Incero - services in that DC and on the hardware involved have been extremely stable, but the underlying software (Linux operating system)needed to be upgraded during a somewhat inopportune moment.  The software "kernel" is now running stable, and should be for the foreseeable future.


If you have questions about this or any other issue, feel free to e-mail us or open a support ticket.

Resolved: Power Issues (Atlanta)

Our server is coming back online, and all services should be available shortly.  Thank you again for your patience.

Power issues affecting one server (Atlanta)

One of our main servers in our Atlanta DC is down, affecting some clients, especially those who access webmail and our forms-based services.  Actual mail delivery is not affected, as mailboxes are on different servers.


The note from the DC is: "Racks L11-20 & M11-20 have lost power due to bus failure."


Services will be up again as soon as the appropriate power is restored.  Thank you for your patience.

Server issues in one datacenter (Canada)

A power issue in the Fibernetics  Telecomm Datacenter in Canada where we host some ancillary services* is causing these systems to remain offline.  According to outage report here:
http://outages.kwhydro.ca/gridvu/
all systems should be back online by this afternoon, based on the Eastern Standard time zone.


* Affected services include:


Canada Proxy and Backup
POP3 to Webmail gateway
Remote office, virtual desktop


If you have any questions or concerns, please open a support ticket or send us an e-mail.  Apologies for any inconvenience.

Resolved: Packet loss at Atlanta Datacenter

Our primary web server has now reconnected and is operating properly; however, there is still one backup system which has not yet recovered.  We are in contact with the DC and will provide an update here only if we need to take additional steps to manage customer data or notify them.  Thank you as always for your patience.

Packet loss at Atlanta Datacenter

An investigation is ongoing at our Atlanta, GA datacenter where we keep our primary webserver as well as some backup systems.  All content and hardware is fine, but there are performance issues concerning inbound packet loss.  We will update here if we have anything concrete to report.  Services and sites may operate intermittently until a fix is completely implemented, and we apologize for the inconvenience. 

Everything old is new again - webmail rollback

In an effort to simplify things for our webmail customers who had grown accustomed to our pre-2017 webmail client, we have reverted back to that system for good.  We have also arranged some new mail storage and hosting options through our European data centers.  Please let us know if you are interested in these privacy-enhanced options.