[Monitoring] Login issues

We are aware that customers can’t log in to their accounts. This is a database related issue and we are looking at this now. We are queuing incoming mail and it will be delivered to accounts once we have resolved this problem. Sorry for the inconvenience.

Update 14-May-2019 13:55 CEST: We are currently experiencing issues with a database system and are working to bring up a replacement, replica system. Our system administrators are working hard to resolve the problem and restore access as quickly as possible, but it might take some time. We will provide further updates as soon as we have them.

Update 14-May-2019 15:44 CEST: We’re having progress with the replica replacement database system. Meanwhile we are working on another contingency plan in case it becomes necessary.

Update 14-May-2019 16:41 CEST: The replica database system has been successfully expanded and prepared, and we are now attempting the same procedure on the production database system in order to resume services as smoothly as possible.

Update 14-May-2019 17:31 CEST: Preparing the production database system with the same procedure that was tested on the replica system was not successful, so we are proceeding with the replica system instead. We are taking steps to ensure that we have additional contingency plans ready.

Update 14-May-2019 18:35 CEST: We have moved the replica database system into production and are now attempting to bring services gradually online. Services are responding and data should be consistent and complete as of 12:04 CEST. Incoming email in the meantime has been queued on our servers and is now being delivered.

Update 14-May-2019 22:22 CEST: Access to email accounts should now be back to normal. Email that was queued has been delivered to accounts. At the moment you will not be able to access your log in details in the Account Security (Access Control) section as these are currently disabled while we monitor the database performance.

Update 16-May-2019 21:00 CEST: We are experiencing problems with the database system and are working to resolve these now.

Update 16-May-2019 22:30 CEST: All services are up and operating normally and all incoming, queued email has been processed. The cause of this disruption was a configuration issue with the replacement database server that was deployed 2 days ago. This has now been corrected and we don’t expect any further issues. We apologize for the inconvenience caused.

Update 21-May-2019 06:30 CEST: We are experiencing login problems and are investigating the issue.

Update 21-May-2019 07:20 CEST: We have found the cause of this error and are implementing a fix.

Update 21-May-2019 07:23 CEST: All services are now coming back online. We apologize for the inconvenience caused.

Posted in Service Status