Nexus Bytes System Heart Beat Status - Incident history

NLVMNode2 - Storage Failure

Sat, 18 Dec 2021 18:10:06 +0000

Dec 18, 18:10:06 GMT+0
Investigating - We have a potential disk failure on the above mentioned node. We are currently investigating this incident and working on bringing everything back up as fast as we can and precent data loss. .

Dec 19, 08:05:33 GMT+0
Resolved - We just resolved the issue! Thankfully no data loss :). 100% SLA Credit has been applied to every single effected family members. :).

NYVM4 - Power outage

Tue, 16 Nov 2021 00:37:21 +0000

Nov 16, 00:37:21 GMT+0
Investigating - We are currently investigating this incident..

Nov 16, 17:18:56 GMT+0
Resolved - We just resolved the issue! Thank you for your patience. We have fixed the issue with the server and is back online now. We have replaced the Liquid cooler and the server seems to be stable now. Please monitor the server and let us know if the issue persists. SLA credit we’re issued to impacted users. .

Hard drive failure: Miami node-4 and Miami node-9

Wed, 27 Oct 2021 19:55:02 +0000

Oct 27, 20:03:21 GMT+0
Monitoring - Node-4 disk has been replaced. Node-9 being worked on..

Oct 27, 19:55:02 GMT+0
Identified - We are continuing to work on a fix for this incident..

Oct 27, 20:10:41 GMT+0
Resolved - We just resolved the issue! Both disks has been swapped. Raid rebuild in progress. 0 data loss. Phew..

NYVM1 Node offline. Issue with Virtualizor - NYC

Mon, 11 Oct 2021 03:17:57 +0000

Oct 12, 03:01:32 GMT+0
Resolved - We just resolved the issue!.

Oct 11, 03:17:57 GMT+0
Investigating - We are currently investigating this incident with virtualizor support team. Only 1 node impacted..

Nyc power outage

Tue, 28 Sep 2021 18:51:50 +0000

Oct 12, 03:01:17 GMT+0
Resolved - We just resolved the issue!.

Sep 28, 18:51:50 GMT+0
Investigating - We are currently investigating this incident..

Sep 28, 22:37:38 GMT+0
Monitoring - We implemented a fix and currently monitoring the result..

Emergency Maintenance Notification: London

Thu, 9 Sep 2021 23:01:12 +0000

Sep 14, 12:59:37 GMT+0
Resolved - We just resolved the issue!.

Sep 9, 23:01:12 GMT+0
Identified - We believe currently that there is a fault in the firmware of the switch that your server is connected to resulting in strange behaviour that can fail to route IPs intermittently; we have schedule an emergency maintenance to upgrade the firmware in an attempt to rectify the problem. .

Sep 10, 00:14:35 GMT+0
Resolved - We just resolved the issue!.

Sep 14, 00:16:10 GMT+0
Identified - Regrettably, following this firmware upgrade, we've observed continued disruption with routing, resulting in infrequent high levels of packet loss - although this is a different symptom that customers previously experienced, this has identified that the fault is not with software, and is likely a hardware fault that has developed. As the packet loss has continued to increase in frequency, we've taken the pre-emptive measure to perform a complete replacement of the device in an attempt to limit the disturbance to your service moving forward. This replacement will be classified as priority maintenance that will be scheduled within the next two hours; we sincerely apologise for the short notice but hope that you'd prefer us to resolve this quickly before it increases its impact or results in a total outage. We don't expect an extended outage; however, we anticipate devices will experience a connectivity disruption of approximately one minute while patch cables are migrated between devices..

Singapore isn't accessible at the moment (automated)

Wed, 21 Jul 2021 06:05:32 +0000

Jul 21, 06:05:32 GMT+0
Investigating - Singapore cannot be accessed at the moment. This incident was created by an automated monitoring service..

Jul 21, 06:16:31 GMT+0
Resolved - Singapore is now operational! This update was created by an automated monitoring service..

Jul 21, 14:24:16 GMT+0
Resolved - Resolved - A network device in Singapore that terminates a portion of BGP sessions experienced TCAM allocation issues. We have implemented a fix. Our Singapore presence has grown rapidly and as we have now experienced stability issues twice in the same month due to the same device having a resource issue we are bring forward planned works to overhaul our Singapore network presence. This work will now start immediately and includes adding multiple cores in seperate data centers, with a new Equinix SG3 network PoP build already in process, upgraded metro connectivity redundancy, capacity with additional T1 carriers and most importantly removing as many single points of failure as possible. Sorry for any inconvenience caused. .

Germany isn't accessible at the moment (automated)

Mon, 5 Jul 2021 20:47:59 +0000

Jul 5, 21:00:35 GMT+0
Resolved - Germany is now operational! This update was created by an automated monitoring service..

Jul 5, 20:47:59 GMT+0
Investigating - Germany cannot be accessed at the moment. This incident was created by an automated monitoring service..

Germany isn't accessible at the moment (automated)

Mon, 14 Jun 2021 19:37:48 +0000

Jun 14, 19:37:48 GMT+0
Identified - Germany cannot be accessed at the moment. This incident was created by an automated monitoring service..

Jun 14, 19:45:27 GMT+0
Resolved - Germany is now operational! This update was created by an automated monitoring service..

Netherlands is inaccessible at the moment (automated) - NLKVM7

Sat, 29 May 2021 12:06:39 +0000

May 29, 12:06:39 GMT+0
Identified - Netherlands cannot be accessed at the moment. This incident was created by an automated monitoring service..

May 29, 14:57:36 GMT+0
Resolved - We just resolved the issue!.

NYC isn't accessible at the moment (automated)

Fri, 21 May 2021 02:47:58 +0000

May 21, 04:56:15 GMT+0
Resolved - Everything is back to normal! .

May 21, 02:47:58 GMT+0
Identified - NYC cannot be accessed at the moment. This incident was created by an automated monitoring service..

May 21, 04:20:37 GMT+0
Monitoring - PNY1 has been restored. NYVMNode-12 will be online shortly..

May 21, 02:49:32 GMT+0
Investigating - NYVMNode-12, NYVMNode14 and PNY1 suffered unexpected system reboot due to drive conflict. We are working on it. .

May 21, 03:10:55 GMT+0
Identified - NYVMNode-14 has been restored. Still working on NYVMNode-12 and PNY1..

Tokyo services network degraded

Wed, 19 May 2021 12:52:27 +0000

May 19, 12:52:27 GMT+0
Investigating - We are currently investigating this incident..

May 19, 12:59:46 GMT+0
Investigating - We are working with an upstream partner and our technicians to determine the cause of the problem and will update further as soon as possible..

May 19, 13:44:11 GMT+0
Resolved - We have patched core router bug and currently monitoring the result..

Miami Storage node 5 is inaccessible at the moment (automated)

Sun, 16 May 2021 11:54:55 +0000

May 16, 11:54:55 GMT+0
Identified - Miami storage node 5 cannot be accessed at the moment. This incident was created by an automated monitoring service..

May 16, 12:47:29 GMT+0
Resolved - Miami is now operational! This update was created by an automated monitoring service..

UK-Storage-5

Sun, 9 May 2021 17:16:01 +0000

May 9, 17:16:01 GMT+0
Identified - We are experiencing brief outage due to faulty memory stick. We are working on it. Only impacts select storage VPS users. .

May 9, 18:24:48 GMT+0
Resolved - We just resolved the issue!.

Nexus Bytes: Scheduled Maintenance Notification: London

Wed, 5 May 2021 06:00:00 +0000

May 5, 06:00:00 GMT+0
Identified - Hi there! This is to inform you that there is a scheduled network maintenance activity which will impact one or more services that you currently have active with us. Please note that your service(s) may not be offline throughout the full duration of the maintenance window. Scheduled Maintenance Activity Details: - Reason: Infrastructure Upgrade - Location/Site Impacted: London - Date: Wednesday 5th May 2021 - Maintenance Window: 02:00 (2AM) - 5:00 (5AM) (EST) - Estimated Power Outage Duration: < 10 Minutes - Estimated Network Outage Duration: < 45 Minutes Maintenance Description: As part of our continued growth and efforts to maintain long-term service sustainability and reliability for you, our customer, we've assumed a revised contract with our Datacenter partner in London that unfortunately requires a portion of our existing infrastructure to be relocated to a different part of the facility. Your server / the host machine your service is run off of is included in this portion. The physical move of assets will result in this maintenance impacting both power and network for the duration that the device is being moved. All services will be gracefully shut down before their migration and started back up once relocated. Action Needed By You: Only impacts our London based services. Please ensure that any mission-critical services have been configured to automatically start before the maintenance window to prevent extended service disruption to your customers. Alternatively, be prepared to start up any required services after a graceful restart as needed. No configuration changes are required. All other customers do not need to perform any actions. Please keep backup as always :) We sincerely apologize for any inconvenience this will cause and if you have any additional follow-up questions related to this maintenance activity, please contact us and we'll be happy to help! Rest assured, we'll try our best to keep any downtime during the maintenance window as minimal as possible. ------------- Thank you for being a member of the Nexus Bytes Family. Nexus Bytes LLC - Affordable boutique servers, priceless bespoke experience. Follow us on Twitter @NexusBytesLLC | Facebook @NexusBytesLLC.

May 5, 11:42:42 GMT+0
Completed - Maintenance has completed successfully..

Miami is inaccessible at the moment (automated)

Fri, 30 Apr 2021 01:50:23 +0000

Apr 30, 01:52:52 GMT+0
Resolved - Miami is now operational! This update was created by an automated monitoring service..

Apr 30, 01:50:23 GMT+0
Identified - Miami cannot be accessed at the moment. This incident was created by an automated monitoring service..

Miami is inaccessible at the moment (automated)

Wed, 14 Apr 2021 22:14:21 +0000

Apr 14, 22:14:21 GMT+0
Identified - Miami cannot be accessed at the moment. This incident was created by an automated monitoring service..

Apr 14, 22:39:08 GMT+0
Resolved - Miami is now operational! This update was created by an automated monitoring service..

Miami is inaccessible at the moment (automated)

Sat, 10 Apr 2021 16:59:23 +0000

Apr 10, 17:11:55 GMT+0
Resolved - Miami is now operational! This update was created by an automated monitoring service..

Apr 10, 18:19:44 GMT+0
Resolved - We just resolved the issue! Replaced faulty network card for hostnode miamivm7.

Apr 10, 16:59:23 GMT+0
Identified - Miami cannot be accessed at the moment. This incident was created by an automated monitoring service..

Japan node 4 is inaccessible at the moment (automated)

Fri, 9 Apr 2021 07:12:32 +0000

Apr 9, 07:12:32 GMT+0
Identified - Japan cannot be accessed at the moment. This incident was created by an automated monitoring service..

Apr 9, 08:15:40 GMT+0
Resolved - Japan is now operational! This update was created by an automated monitoring service..

Apr 9, 13:03:22 GMT+0
Identified - We are continuing to work on a fix for this incident..

Apr 9, 14:07:44 GMT+0
Resolved - Japan is now operational! This update was created by an automated monitoring service..

Netherlands is inaccessible at the moment (automated) - NLpVM2

Wed, 31 Mar 2021 17:25:12 +0000

Mar 31, 17:25:13 GMT+0
Identified - Netherlands cannot be accessed at the moment. This incident was created by an automated monitoring service..

Mar 31, 17:34:07 GMT+0
Identified - Power failure due to a PDU outage. Our engineers are currently replacing the PDU and to restore power ASAP. You will be eligible for SLA Credit, once the service comes back up. It will be automated and there won't be a need to open ticket..

Mar 31, 17:50:30 GMT+0
Resolved - We just resolved the issue!.

Scheduled Maintenance - March 27, 2021

Sat, 27 Mar 2021 22:00:00 +0000

Mar 27, 22:00:00 GMT+0
Identified - Dear Users, This is to notify you of an upcoming scheduled system maintenance. **Reason:** Standard hardware maintenance and system improvement. **Date:** March 27th, 2021 **Time:** 6:00 P.M to 11:59 P.M (EST) **Expected maximum downtime per service:** 30 minutes **Impacted Services:** All Services in the following region: - **NYC Metro, Miami, Los Angeles, London, Singapore, Tokyo** Note: Storage VPS's won't be impacted. If you have any questions or concerns, please [contact us here](https://my.nexusbytes.com/submitticket.php). If your service do not come back online at the end of maintenance period, please open support ticket immediately. Please check our Heartbeat monitor (https://heartbeat.nexusbytes.com/) to stay updated. We do apologies for any inconvenience this might have caused you and we appreciate you for being part of the Nexus Bytes family. **P.S: While we do not anticipate any data loss during the maintenance period, however, as always, please please and please keep an offsite backup of your data.**.

Mar 27, 22:22:28 GMT+0
Identified - Maintenance is now in progress..

Mar 27, 23:30:03 GMT+0
Completed - Maintenance has completed successfully..

Emergency Maintenance Notification: Miami Storage node 2

Mon, 15 Mar 2021 17:00:00 +0000

Mar 15, 17:00:00 GMT+0
Identified - Dear Users, This is to notify you of an upcoming scheduled emergency system maintenance. Reason: Replacing failing memory module Date: March 15th, 2021 Time: 1:00 P.M to 2:30 P.M (EST) Expected downtime: 90 minutes Impacted Services: Storage VPS hosted on Miami Storage node 2 If you have any questions or concerns, please [contact us here.](https://my.nexusbytes.com/submitticket.php) We do apologies for any inconvenience this might have caused you and we appreciate you for being part of the Nexus Bytes family. .

Mar 15, 20:27:41 GMT+0
Completed - Maintenance has completed successfully..

Germany is inaccessible at the moment (automated)

Wed, 3 Mar 2021 03:06:29 +0000

Mar 3, 03:06:29 GMT+0
Identified - Germany cannot be accessed at the moment. This incident was created by an automated monitoring service..

Mar 3, 03:08:11 GMT+0
Resolved - We just resolved the issue!.

Mar 3, 03:14:03 GMT+0
Resolved - Germany is now operational! This update was created by an automated monitoring service..

Nexus Bytes: Emergency System Maintenance Notification : February 28th, 10:30 PM (EST) {NY Node 15}

Mon, 1 Mar 2021 03:30:00 +0000

Mar 1, 03:30:00 GMT+0
Identified - We have detected hardware issue on the node. We will be taking the host node down for ~ 15 minutes to swap out the faulty disk. .

Mar 1, 03:48:14 GMT+0
Completed - We have successfully replaced the faulty hard drive. Thank you for your patience. .

Psg1- Degraded performance.

Fri, 26 Feb 2021 12:10:09 +0000

Feb 26, 12:10:09 GMT+0
Investigating - We are currently investigating Singapore shared hosting node: psg1 This has been causing multiple downtime over the last 3 days. .

Feb 28, 02:58:00 GMT+0
Resolved - We have successfully resolved this issue..

Germany storage node 5 - Emergency HW replacement

Thu, 11 Feb 2021 20:38:55 +0000

Feb 11, 20:38:55 GMT+0
Identified - - Detected faulty HDD. - Being replaced. - Approximate downtime: 15 minutes. - Expect intermediate performance issue as the raid rebuilds. - Should go back to normal within 24 hours. .

Feb 11, 20:46:59 GMT+0
Monitoring - .

Feb 20, 22:12:43 GMT+0
Resolved - We just resolved the issue!.

london1kvm - node rebooted to replace faulty memory module.

Sat, 2 Jan 2021 16:41:59 +0000

Jan 2, 16:55:44 GMT+0
Resolved - We just resolved the issue!.

Jan 2, 16:41:59 GMT+0
Identified - We have identified faulty ram stick on our london1kvm node (UK). Node rebooted to replace the faulty HW. .

NYC is inaccessible at the moment (automated)

Mon, 21 Dec 2020 03:26:20 +0000

Dec 21, 03:26:20 GMT+0
Identified - NYC cannot be accessed at the moment. This incident was created by an automated monitoring service..

Dec 21, 03:43:09 GMT+0
Resolved - NYC is now operational! This update was created by an automated monitoring service..

NYC is inaccessible at the moment (automated)

Wed, 2 Dec 2020 20:18:42 +0000

Dec 2, 20:18:43 GMT+0
Identified - NYC cannot be accessed at the moment. This incident was created by an automated monitoring service..

Dec 2, 20:23:34 GMT+0
Resolved - NYC is now operational! This update was created by an automated monitoring service..

Dec 2, 20:21:37 GMT+0
Monitoring - nyvm7 had a temporary power outage, resulting in total of 2 minutes downtime. We are currently investing the root cause..