[Resolved] Access issues

We are currently experiencing problems that might mean your access to webmail, IMAP and POP services are slow or not possible at all. We are investigating this and will update this post as we know more. Sorry for any inconvenience.

Update 2020-04-23 23:45 CEST: We are analyzing file system and network issues in order to resolve the access issues, while working to tune the system better to alleviate the immediate problems.

UPDATE 2020-04-24 05:45 CEST: The problems seem caused by a malfunctioning storage unit affecting access. We are continuing to investigate the underlying cause.

UPDATE 2020-04-24 08:50 CEST: We are attempting an upgrade of the storage unit that is having problems.

UPDATE 2020-04-24 09:15 CEST: We suspect faulty hardware in one of our storage units and are now restarting it while initiating acquisition of a replacement server.

UPDATE 2020-04-24 09:45 CEST: The restart of the storage units and all services that utilize it does not seem to have helped. We are now considering hardware replacements in the storage unit itself.

UPDATE 2020-04-24 10:25 CEST: We are in communication with our hardware supplier regarding replacement parts for the storage unit, and have an alternate plan for acquiring required hardware if needed.

Our sysadmin team have established plans for testing hardware replacement and operating system replacement. Additionally they are developing plans for replacement of SSD disks if that will be necessary.

UPDATE 2020-04-24 11:00 CEST: A full replacement server is being acquired in order to execute the replacement as efficiently as possible.

UPDATE 2020-04-24 13:15 CEST: Following a restart of the malfunctioning storage unit, load on application servers have normalized. We are still pursuing a plan to migrate data away from this storage unit through today and the weekend.

UPDATE 2020-04-24 18:20 CEST: The replacement storage unit as well as a new backup server are installed in our data center.

UPDATE 2020-04-24 19:50 CEST: The new storage unit is being set up and configured.

UPDATE 2020-04-25 10:15 CEST: Migration of data from the storage unit with access issues to the replacement server has commenced.

UPDATE 2020-04-27 01:36 CEST: We are experiencing issues with the storage unit we are working to migrate away from and subsequent access problems especially via IMAP. Our system administrators are working to resolve the issue. We apologize for the inconvenience caused.

UPDATE 2020-04-27 09:15 CEST: After further analysis and configuration tuning the IMAP/POP services are stabilized and performing normally.

UPDATE 2020-04-29 09:25 CEST: All services are continuing to operate normally. We are proceeding with the migration to the new server with encrypted storage.


Posted

in

by

Tags: