PulseHeberg - Incident DC Nice – Incident details

Incident DC Nice

Resolved
Operational
Started 1 day agoLasted about 7 hours

Affected

Our datacenters

Partial outage from 1:35 PM to 8:14 PM

🇫🇷 Nice-Sophia Antipolis (NCE)

Partial outage from 1:35 PM to 8:14 PM

Hosted infrastructures

Partial outage from 1:35 PM to 8:14 PM

Virtual servers

Partial outage from 1:35 PM to 8:14 PM

Baremetal servers

Partial outage from 1:35 PM to 8:14 PM

Updates
  • Resolved
    Resolved

    The incident has been fully resolved. Depending on the duration of the incident that impacted your service, you may be eligible for SLA compensations upon request to support.

  • Update
    Update

    The dedicated ECO servers are back online. All VPS Performance services and dedicated servers are operational. Our team is finalizing the restart of the VPS Education servers for IUT Troyes and Belfort Montbéliard.

  • Update
    Update

    The majority of affected services have been restored. VPS Performance servers and SD06 dedicated servers are all back online as of about 20 to 30 minutes ago. Our teams are finalizing the restart of the ECO dedicated servers affected by the incident as well as some Education Cloud VPS in partnership with the IUTs of Troyes and Belfort-Montbéliard.

  • Update
    Update

    The Nice teams arrived in Marseille about fifteen minutes ago, and the racking of the machines is currently underway.

  • Update
    Update

    Our team arrived on site and performed a series of diagnostics on the blade center. The fault is located on the blade center's backplane; this part must be replaced to restart the blade center and the affected services.

    We have this component in spare stock at our Marseille DC, but its delivery would require approximately 4 hours (round trip Nice-Marseille) as well as approximately 30 additional minutes to disassemble and reassemble the component.

    Due to the small number of physical servers impacted, and to reduce downtime, we have chosen to bring forward the move of these servers to Marseille, initially planned for May. This allows us to cut the recovery estimate in half by making a one-way trip to Marseille (2 hours).

    The move is underway, we estimate the resumption of services around 6:30 p.m.

  • Identified
    Identified

    We are experiencing an incident on one of our blade centers, hosting around twenty servers (dedicated servers, as well as VPS nodes).

    Despite the blade center's fully redundant design, it appears that an electrical fault caused all machines located on the blade center to completely shut down.

    One of our teams is en route to the DC to perform advanced hardware diagnostics and explore the fastest recovery options for the affected servers.

    We expect the next update on the situation within 30 minutes.

  • Investigating
    Investigating

    Since 3:35 p.m. we have observed unavailability of certain services hosted on the Nice data center, affecting dedicated servers (ECO and SD06 range), as well as VPS from the Performance range.

    Our team is working to identify the cause of this unavailability. We expect to have an initial update on the situation within 15 minutes.