Ultime informazioni sulla situazione a Strasburgo

Roubaix, Francia | 18-03-2021

Ultime informazioni sulla situazione a Strasburgo

Strasbourg datacentre: latest information

Roubaix, France | 18 March 2021 - 16:30 (CET)

Customer information:

1. Service Status

  • In the OVHcloud Control Panel, each affected customer can view the status of their services located in Strasbourg (SBG) - as detailed in the FAQ.
  • The OVHcloud Community platform is now available to interact with community members and support you.
  • By following the security protocol defined for on-premises operations: the restart will be carried out room by room, aisle by aisle and rack by rack.
  • Procedures for restarting services will be communicated to affected customers as details become available in the coming days.
  • Real-time monitoring of the restart of SBG by room and planned rack is available at: status.ovh.com/vms/sbg.

2. Commercial actions

  • At the beginning of the week, all customers concerned received a communication on the commercial measures that will be applied to their situation. For customers who have not received the email, a procedure has been indicated in the FAQ.

3. Plan of action

Priority 1: restoring services to SBG1, SBG3 and SBG4

  • Status summary for each datacentre

Datacentre

Situation

Electrical restart

Network restart

Server restart

SBG-1

Datacentre operational.

 

Accomplished

Accomplished

First racks tested with success.

Handover to the customer today with visibility of the status in their OVHcloud Customer Control Panel.

SBG-2

Building non-operational.

Site secured.

 

 

Replacement of infrastructure in other datacentres: Gravelines (GRA), Roubaix (RBX), London (LON), Warsaw (WAW), Frankfurt (FRA)

SBG-3

Datacentre operational.

 

Servers undamaged.

 

 

 

Accomplished

 

Accomplished

Test today on one rack and if conclusive provisional estimate for Friday 19 March for a progressive restart of the services.

SBG-4

Datacentre operational.

 

All floors and servers undamaged.

Accomplished

 

 

Accomplished

 

Provisional ETA: Monday, 22 March for gradual restart of services.

 

  • 136 people are mobilised on premises today
    • The team rotation schedule for the coming weekend has been established.
  • Cleaning of site:
    • Cleaning teams are active in SBG1 and SBG3 in all affected areas.
    • Some of the servers in SBG3 require specific cleaning prior to their return to service. We have started this process.

Priority 2: Provisioning of infrastructure for our customers in other datacentres

  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our other datacentres in Gravelines (GRA), Roubaix (RBX), Londres (LON), Warsaw (WAW), and Frankfurt (FRA).
  • We are committed to providing nearly 15,000 new servers to our customers in the coming weeks.
  • Update on quantities delivered since 10th March:

CLOUD UNIVERSE

PRODUCTS

 

QUANTITY

Bare metal

Bare metal

Servers

4519

 

NAS HA

TB

246

Public Cloud

VPS

VM

12353

Hosted Private Cloud

Host

Servers

485

 

Datastore

Number of zpool

522

 

 

Volume of TB

3067

  • 400 servers will be produced today.

Due to the current demand, delivery times on our Bare Metal services may be longer than usual. Our teams are fully mobilised, and we are working hard to deliver to our customers as quickly as possible, especially all affected customers.

On the other hand, Public Cloud, Web Cloud and VPS services are being delivered as normal in all datacentres. 

Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers

  • Our Support teams and Technical Account Managers are in contact with our customers and have set up a support plan for customers who have not yet activated their DRP.

Communications

  • The OVHcloud Community platform is now available to interact with community members and support you.
  • An FAQ is available here.
  • To follow updates about this situation in real time:
    • Our main communication tool remains "Task Status"
    • Status videos are available here:

- FR : https://twitter.com/olesovhcom/status/1371908744775360518

- EN : https://twitter.com/olesovhcom/status/1371934994751500288

Our mission remains to provide our customers with the highest quality of service, to support their online business.  and we know how important this is to them. We sincerely apologise any inconveniences related  this situation An investigation into the cause is underway. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

-----------

17 March 2021 - 16:30 (CET)

Customer information:

1. Service Status

  • Octave Klaba, founder of OVHcloud, has released a new video on the situation in Strasbourg (SBG).
  • In the Control Panel, each affected customer can view the status of their services located in Strasbourg (SBG) - as detailed in the FAQ.
  • The OVHcloud Community platform is now available to interact with community members and support you.

By following the security protocol defined for on-premises operations: restarting will be carried out room by room, aisle by aisle and rack by rack.

  • Procedures for restarting services will be communicated to affected customers as details become available in the coming days.
  • Real-time monitoring of SBG restart by room and planned rack via: status.ovh.com/vms.
  • Progressive updates to the status page for services and backups in Strasbourg (SBG), from today onwards, accessible via the FAQ.
  • Updating and tracking the percentage of services restarted per datacentre scheduled in the “Task Status” interface in the coming days.
  • A new order form has been set up (for Bare Metal Cloud) to offer concerned customers services similar to those they had at the Strasbourg (SBG) site.

2. Commercial actions

  • All concerned customers received, at the beginning of the week, a communication on the commercial measures that will be applied to their situation. For customers who have not received the email, a procedure has been indicated in the FAQ.
  • For each customer concerned:
    • A billing suspension notification for the month of March has been added in the Customer Control Panel.
    • Commercial actions have been indicated, the details of which will be communicated this week.

3. Plan of action

Priority 1: restoring services to SBG1, SBG3 and SBG4

  • Status summary for each datacentre

Datacentre

Situation

Electrical restart

Network restart

Server restart

SBG-1

4 of 12 rooms were damaged

 

- Rooms 61AD/62AD: servers not affected

- Rooms 61E/62D: cleaning of servers underway

- Restored power to datacentre

 

- Backbone: reconnected

- Redeployment of the local network (Cabling finalised this Wednesday) and configuration today

 

 

A rack was tested today and if conclusive a provisional estimate of Friday, 19 March for a gradual restart of services after cleaning and drying servers.

 

SBG-2

Building out of use Site secured

 

 

Replacement of infrastructure in other data centres: Gravelines (GRA), Roubaix (RBX), London (LON), Warsaw (WAW), Frankfurt (FRA)

SBG-3

Servers undamaged

 

 

 

- Restored power to datacentre

 

 

- New network power restored

- Redeployment of the local network this Wednesday 17th March (Cabling finalised).

 

Test tomorrow on a rack and if conclusive a provisional estimate of Friday, 19 March for a gradual restart of services.

SBG-4

All floors and servers were spared

- Restored power to datacentre

 

Redeployment of the local network carried out.

Provisional ETA: Monday, 22 March for gradual restart of services.

 

  • 116 people are mobilised at the premises today
  • Network redeployment: 
    • The SBG1, SBG3 and SBG4 datacentres will be connected to the network tonight.
  • Watercooling:
    • The entire system is operational in the SBG1/SBG3/SBG4 datacentres.
  • Cleaning of site:
    • Intensive mobilisation (24/24).
    • SBG1: finalisation of cleaning Wednesday, 17th March.

Priority 2: providing infrastructures in other datacentres for our affected customers

  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our other datacentres in Gravelines (GRA), Roubaix (RBX), Londres (LON), Warsaw (WAW), and Frankfurt (FRA).
  • We are committed to providing nearly 15,000 new servers to our customers in the coming weeks. Update on quantities delivered since 10th March:

CLOUD UNIVERSE

PRODUCTS

 

QUANTITY

Bare metal

Bare metal

Servers

4200

 

NAS HA

TB

276

Public Cloud

VPS

VM

11998

Hosted Private Cloud

Host

Servers

434

 

Datastore

Number of zpool

466

 

 

Volume of TB

2950

 

  • 400 servers will be produced today.

Due to the current demand, delivery times on our Bare Metal services may be longer than usual. Our teams are fully mobilised, and we are working hard to deliver to our customers as quickly as possible, especially all affected customers.

On the other hand, Public Cloud, Web Cloud and VPS services are being delivered as normal in all datacentres. 

Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers

  • Our Support teams and Technical Account Managers are in contact with our customers and have set up a support plan for customers who have not yet activated their DRP.

Communications

- FR: https://twitter.com/olesovhcom/status/1370045226929299461

- EN: https://twitter.com/olesovhcom/status/1370045891806175232

Our mission remains to provide our customers with the highest quality of service, to support their online business.  and we know how important this is to them. We sincerely apologise any inconveniences related  this situation An investigation into the cause is underway. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

------

16 March 2021 - 16:00 (CET)

Customer information:

1.  Service Status

  • Progressive updates to the status page for services and backups in Strasbourg (SBG), from today onwards, accessible via the FAQ.
  • Monitoring of the % of service restarts per datacentre to be displayed in the "Travaux" interface in the coming days. Restarts will be carried out room by room, aisle by aisle and rack by rack.
  • Finalized procedures for restarting services will soon be communicated to customers.

2. Commercial actions

  • For each customer concerned:
    • A billing suspension notification for the month of March has been added in the Customer Control Panel.
    • Commercial actions have been indicated, the details of which will be communicated this week.

3. Plan of action

Priority 1: restoring services to SBG1, SBG3 and SBG4

  • Status summary for each datacentre

Datacentre

Situation

Electrical restart

Network restart

Server restart

SBG-1

4 of 12 rooms were damaged

 

- Rooms 61AD/62AD: servers not affected

- Rooms 61E/62D: cleaning of servers underway

- Power restored

- Damaged UPS have been checked and are being replaced

- Backbone: temporary connection to SBG on 15th March

- LAN redeployment and configuration today

 

 

Provisional ETA: Monday, 22 March for gradual restart

SBG-2

Building out of use. Structural expertise carried out. Site secured.

 

 

 

 

Infrastructure replacement in other data centres Gravelines (GRA), Roubaix (RBX), London (LON), Warsaw (WAW), Frankfurt (FRA)

SBG-3

Servers undamaged

 

 

- Power restored today

- All inverters are operational

 

 

- New network room powered up today

- Redeployment of the local network on 17th March

Provisional ETA: Monday, 22 March for gradual restart

 

 

SBG-4

All floors and servers were spared

Power restored today

 

Redeployment of the local network today

Provisional ETA: Monday, 22 March for gradual restart

 

  • 101 people are mobilised and on premises
  • Network redeployment:
    • SBG connection to the backbone today (alternative routes to Paris and Frankfurt underway)
    • Reconnection of buildings and rooms:
      • Fibre optic redeployment in progress 
      • Configuration underway
  • Watercooling:
    • The entire system is operational.
  • Cleaning of site:
    • Intensive mobilisation (24/24).

Priority 2: Provisioning of infrastructure for our customers in other datacentres

  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our other datacentres: Gravelines (GRA), Roubaix (RBX), Londres (LON), Warsaw (WAW), Francfort (FRA).
  • We are committed to providing nearly 15,000 new servers to our customers in the coming weeks. Update on quantities delivered since 10th March:

CLOUD UNIVERSE

PRODUCTS

 

QUANTITY

Bare metal

Bare metal

Servers

3939

 

NAS HA

TB

242

Public Cloud

VPS

VM

10731

Hosted Private Cloud

Host

Servers

372

 

Datastore

Number of zpool

393

 

 

Volume of TB

2704

 

  • 350 servers will be produced today.

Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers

  • Our Support teams and Technical Account Managers are in contact with our customers and have set up a support plan for customers who have not yet activated their DRP.

Communications

- FR: https://twitter.com/olesovhcom/status/1370045226929299461

- EN: https://twitter.com/olesovhcom/status/1370045891806175232

Our mission remains to provide our customers with the highest quality of service, to support their online business and we know how important this is to them. We sincerely apologise for any inconveniences related to this situation. An investigation into the cause is underway. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

----------

15 March 2021 - 17:00 (CET)

Customer information:

1 - Server status

  • The status page for services and backups in Strasbourg (SBG) is now available in the FAQ.
  • For VPS service, a question about “zone vs datacentre (SBG)” has been added for clarification in the FAQ.

2 - Commercial actions

  • For each customer concerned:
    • A billing suspension notification for the month of March has been added in the Customer Control Panel.
    • Commercial actions have been indicated, the details of which will be communicated this week.

3 - Action plan

Priority 1: restoring services to SBG1, SBG3 and SBG4

  • Status summary for each datacentre

 

Datacentre

Situation

Electrical restart

Network restart

Server restart

SBG-1

4 of 12 rooms were damaged

Temporarily repowered on 13th March and will be restored permanently on 16th March

- Backbone: temporary connection to SBG on 15th March

- Internal network to be redeployed on 16th March

Provisional ETA: Monday, 22 March

for gradual restart

SBG-2

Building out of use

Structural expertise carried out.  Site secured

 

 

Replacement of infrastructure in other datacentres

SBG-3

Servers undamaged

Temporarily repowered on 12th March and will be restored permanently on 16th March

- New network room to be deployed and powered on 16th March

- Internal network to be redeployed on 17th March

Provisional ETA: Monday, 22 March

for gradual restart

SBG-4

Servers undamaged

Power to be restored on 16th March

- Internal network to be redeployed on 16th March

Provisional ETA: Monday, 22 March

for gradual restart

 

  • 111 people are dedicated on-site to connecting network and electricity units (in rotation, 24/24), for: 
  • Power redeployment:
    • The electrical installations required to restart SBG1/SBG4 have been verified.
    • The high-voltage power supply for SBG3 is deployed and operational.
    • Inverter re-commissioning protocol underway today (verification of all UPS).

 

  • Network redeployment:
    • The SBG1 network room is being tested.
    • The new network room is being completed and will be supplied with electricity as soon as the inverters are verified.
    • The installation of the network equipment will be finalised this evening.
    • The connection of the backbone is scheduled for tomorrow.
    • The local network is being redeployed.

 

  • The cleaning of any soot in the SBG3 server rooms will begin on Tuesday, 16 March. The clean-up teams have arrived on site and are beginning deployed.

 

Priority 2: providing infrastructures in other datacentres for our affected customers

  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our Roubaix (RBX) and Gravelines (GRA) datacentres.
  • We are committed to providing nearly 15,000 new servers to our customers in the coming weeks.
  • 500 servers will be produced today.

 

  • Update on quantities delivered since 10th March:

CLOUD UNIVERSE

PRODUCTS

 

QUANTITY

Bare metal

Bare metal

Servers

3658

 

HA-NAS

TB

220

Public Cloud

VPS

VM

8903

Hosted Private Cloud

Host

Servers

328

 

Datastore

Number of zpool

361

 

 

Volume of TB

2598

 

Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers

  • Our Support teams and Technical Account Managers are in contact with our customers, and have set up a support plan for customers who have not yet activated their DRP.

Communications

- FR: https://twitter.com/olesovhcom/status/1370045226929299461

- EN: https://twitter.com/olesovhcom/status/1370045891806175232

 

Our mission remains to provide our customers with the highest quality of services to support their online activities, and we know how important this is to them. We sincerely apologise for the problems caused by this incident. An investigation into the cause is underway. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

--------

14 March 2021 - 15:00 (CET)

 

Customer information:

1 - Server status

  • A page has been created listing the backups of services hosted in Strasbourg (SBG) managed by the customer or by OVHcloud along with their status, which will be available in the FAQ within the next few hours.
  • For Public Cloud and VPS customers, a question about “zone vs datacentre (SBG)” has been clarified in the FAQ today.

2 - Commercial actions

  • All affected customers will be notified about their suspended March bill in their Control Panel.
  • Details on how these measures will be implemented will be sent next week, and will also apply to March bills for impacted services that were already issued before 10th March.

3 - Action plan:

Priority 1: restoring services to SBG1, SBG3 and SBG4

  • Status summary for each datacentre

 

Datacentre

Situation

Electrical restart

Network restart

Server restart

SBG-1

4 of 12 rooms were damaged

Temporarily repowered on 13th March and will be restored permanently on 17th March

- Backbone: temporary connection to SBG on 15th March

- Internal network to be redeployed on 17th March

Provisional ETA: Monday, 22 March

for gradual restart

SBG-2

Building out of use

 

 

Replacing infrastructures in other datacentres

SBG-3

Servers undamaged

Temporarily repowered on 12th March and will be restored permanently on 16th March

- New network room to be deployed and powered on 16th March

- Internal network to be redeployed on 17th March

Provisional ETA: Monday, 22 March

for gradual restart

SBG-4

Servers undamaged

Power will return permanently on 17th March

Internal network to be redeployed on 17th March

Provisional ETA: Monday, 22 March

for gradual restart

 

  • A team of 60 people is working on site to connect mobile network and electricity units (working in rotation, 24 hours a day)
    • Cabling in progress.
    • Connection planned for tonight.
  • The drying out of the SBG3 buildings will be finished tonight.
  • The cleaning of any soot in the SBG3 server rooms will begin on Tuesday, 16 March.
  • The cooling system is operational :
    • Water cooling is protected from the cold weather (forecast locally on Thursday, 18 March and Friday, 19 March).
    • The pumps are operational.
  • Priority 2: providing infrastructures in other datacentres for our affected customers
     
  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our Roubaix (RBX) and Gravelines (GRA) datacentres.
  • We are committed to providing around 15,000 new servers in the coming weeks.
  • Update on quantities delivered since 10th March:

 

Statement

 

  • Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers
    • Our Support teams and Technical Account Managers are in contact with our customers, and have set up a support plan for customers who have not yet activated their DRP.

Communications

- FR: https://twitter.com/olesovhcom/status/1370045226929299461

- EN: https://twitter.com/olesovhcom/status/1370045891806175232

 

Our mission remains to provide our customers with the highest quality of services to support their online activities, and we know how important this is to them. We sincerely apologise for the problems caused by this fire. We will continue to communicate with the utmost transparency about the cause of the fire and its consequences.

An investigation is ongoing. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

------------

13 March 2021 - 16:00 (CET)
 

Customer information:

1 - Server Status

  • We have sent emails to all our customers informing them of the status of their services.

2 - Commercial actions

  • Billing has been suspended for Strasbourg (SBG) datacentre customers from the date of the incident.
  • Commercial measures have been set out to each affected customer.
  • Details on how these measures will be implemented will be sent next week, and will apply in particular to March bills for impacted services that were already issued before 10th March.
  • We have updated the FAQs.

 Our three priorities are as follows:

  • Priority 1: restoring services to SBG1, SBG3, and SBG4
  • Priority 2: providing infrastructures in other datacentres for our affected customers
  • Priority 3: Implementing all DRP (Disaster Recovery Plan) mechanisms with our customers

Action plan:

Priority #1:

  • Status summary for each datacentre

Datacentre

Situation

Electrical restart

Server restart

SBG-1

4 of 12 rooms were damaged

From Monday, 15 March

Progressive restoration of undamaged servers by Monday, 22 March

SBG-2

Building out of use

Audit and inventory of premises

Replacing infrastructures in other datacentres

SBG-3

Servers undamaged

Tests for the high voltage supply will be carried out this weekend.  Provisional restart on 15th March

A progressive restart of all services is estimated from Monday, 22 March.

 

SBG-4

Servers undamaged

Scheduled for Monday, 15 March

A progressive restart of all services is estimated from Monday, 22 March.

 

 

  • Mobile high-voltage network and electricity units arrived at the Strasbourg site today at 12pm.
  • 60 experts are currently working on site (in rotation over the weekend) in four teams, with the following missions:
  1. Team 1 (network) is tasked with connecting the mobile network unit and testing the network room in SBG1
  2. Team 2 (electricity) is working on high-voltage electricity connections
  3. Team 3 (electricity) is working on low-voltage electricity connections
  4. Team 4 is working on protecting and restoring the cooling systems

The aim is to reconnect, repower and restart the backbone access network from Monday, 15 March (following the authorisation of the ESR (Électricité de Strasbourg Réseaux)).

To repower the site:

  • Two generators have been rented since 11th March to power the temporary office.
  • One additional generator was rented to recharge the batteries of our generators following their activation during the incident.

 

  • Several service providers are currently working on site (30 people)
    • The water on the floor of the SBG3 building has been cleaned up
    • The SBG3 building is currently being dried out, which should be completed by Sunday evening
    • The cleaning of any soot in the server rooms of SBG3 will begin on Monday.
       
  • Repair of the damaged cooling system:
    • A crane has arrived on site.

Priority #2:

  • For customers who have been impacted, we are offering replacement infrastructures (Bare Metal, Hosted Private Cloud and Public Cloud) in our Roubaix (RBX) and Gravelines (GRA) datacentres.
     
  • We are committed to providing around 15,000 new servers in the coming weeks.
     
  • Update on quantities delivered since 10th March:
•	Update on quantities delivered since 10/03

Priority #3:

  • Backup mechanisms:
    • An overview of all backup mechanisms will be published in the FAQ today.
    • For Public Cloud and VPS customers, questions about “zone vs datacentre (SBG)” will be clarified in the FAQ today.
    • An ad hoc mechanism is in place to help our customers set up their DRP system.

Communications

- FR: https://twitter.com/olesovhcom/status/1370045226929299461

- EN: https://twitter.com/olesovhcom/status/1370045891806175232

Our mission remains to provide our customers with the highest quality of service, to support their online business. And we know how important this is to them. We sincerely apologise for the problems caused by this fire. We will continue to communicate with the utmost transparency about the cause of the fire and its consequences.

An investigation is ongoing. Whatever the cause of the incident, OVHcloud is currently evaluating potential technical and operational measures to bring solutions to all affected customers.

All of our communication channels, including our incident tracking platform, can be accessed so that you can stay informed of developments in real time.

-------

12 marzo 2021 - ore 14:00 (CET)

Informazioni ai clienti

  • Octave Klaba, fondatore di OVHcloud, ha pubblicato alcuni video sullo stato della situazione:

FR : https://twitter.com/olesovhcom/status/1370045226929299461

EN : https://twitter.com/olesovhcom/status/1370045891806175232

  • Oggi informeremo via email i nostri clienti sullo stato dei loro servizi. Abbiamo anche organizzato un team di supporto 24 ore su 24, 7 giorni su 7 per accompagnarli.

Le nostre tre priorità sono:

  • Priorità 1: ripristino dei servizi su SBG1, SBG3 e SBG4.
  • Priorità 2: messa a disposizione di infrastrutture per i nostri clienti nei datacenter non interessati.
  • Priorità 3: implementazione in collaborazione con i nostri clienti di tutti i meccanismi di DRP (Disaster Recovery Plan).

Priorità 1: ripristino dei servizi su SBG1, SBG3 e SBG4

  • La diagnostica e l’inventario sono in corso.
  • La pulizia e la riparazione degli edifici danneggiati sono in corso per garantire la sicurezza dei nostri team.
  • La pulizia della schiuma e l'aspirazione dell'acqua sulle strade pubbliche sono state effettuate. Il serbatoio di alimentazione del circuito idraulico è stato svuotato.
  • Tutte le connessioni in fibra sono operative.
  • SBG1 è stata parzialmente danneggiata (4 sale su 12)
    • La sala rete è stata controllata ieri. Durante il fine settimana è previsto un test di funzionamento elettrico isolato, senza connettività. Recupero stimatodella connettività: inizio settimana prossima.
    • Il riavvio della rete elettrica é provvisoriamente prevista perlunedì 15 marzo.
    • I server saranno controllati e riattivati uno ad uno.
  • SBG3 è stato risparmiato dall'incendio
  • Entro la fine di questa settimana sarà allestita una sala di rete, che sarà operativa all’inizio della settimana del 15 marzo.
  • Nel fine settimana saranno effettuati i test per l’alimentazione ad alta tensione. Stima provvisoria per il ripristino dell'alimentazione: lunedì 15 marzo.
  • Il riavvio graduale di tutti i servizi è previsto a partire da lunedì 22 marzo.
  • SBG4 è stato risparmiato dall'incendio
    • Completeremo il nostro piano di riattivazione del servizio entro la fine di questa settimana.

Tutti i servizi Email Pro interessati sono stati ripristinati giovedì 11 marzo, a partire dalle 16:55. Nessuna email è andata persa.

Priorità 2: disponibilità di infrastrutture per i nostri clienti nei datacenter che non sono stati colpiti

  • Offriamo ai clienti interessati un'infrastruttura sostitutiva (Bare Metal, Hosted Private Cloud e Public Cloud) nei datacenter di Roubaix (RBX) e Gravelines (GRA).
  • Abbiamo già consegnato ai nostri clienti coinvolti:
    • mercoledì 10 marzo: 2.100 nuovi server
    • giovedì 11 marzo: 630 nuovi server
  • La nuova organizzazione delle nostre catene di montaggio è attiva da venerdì 12 marzo al mattino e permette di triplicare la nostra capacità di produzione (da 3.000 a 4.000 server alla settimana).
  • OVHcloud si impegna a produrre circa 15.000 nuovi server nelle prossime settimane.
  • Comunicheremo al più presto i tempi di consegna dei nuovi servizi.

Priorità 3: implementazione in collaborazione con i nostri clienti di tutti i meccanismi di DRP (Disaster Recovery Plan)

  • I nostri team di supporto e i nostri Technical Account Manager definiranno, entro questo fine settimana, un piano di assistenza destinato ai nostri clienti che non hanno ancora attivato il loro DRP.

Comunicazione

- FR : https://twitter.com/olesovhcom/status/1370045226929299461

- EN : https://twitter.com/olesovhcom/status/1370045891806175232

Il nostro compito è quello di fornire ai nostri clienti la massima qualità di servizio per supportare la loro attività online. E sappiamo quanto sia importante per loro. Ci scusiamo sinceramente per i problemi causati da questo incendio. Ci impegniamo a continuare a comunicare con la massima trasparenza sulla causa dell’incendio e sulle sue conseguenze.

Un'inchiesta è attualmente in corso. Indipendentemente dalla causa dell'incidente, OVHcloud sta attualmente valutando le potenziali misure tecniche e operative volte a fornire soluzioni ai clienti coinvolti.

Tutti i nostri canali di comunicazione, a partire dalla piattaforma di monitoraggio degli incidenti travaux.ovh.com, incluso il task), sono accessibili e aggiornati in tempo reale.

----------

11 Marzo 2021 - 15:30 (CET)
 

Team:

  • I nostri team HR sono arrivati sul posto oggi per dare supporto agli impiegati che stanno lavorando nell’area interessata
  • Sul posto è stato installato un ufficio temporaneo
  • Altri team industriali e tecnici OVHcloud sono arrivati in loco per fornire supporto e competenze specifiche
  • Abbiamo potenziato il personale nel sito di produzione francese (Croix) con volontari interni, per far fronte all’attuale piano di produzione e al carico di lavoro supplementare causato dall’incidente di Strasburgo
  • La produzione dei server si prolungherà oltre in orari di lavoro prolungati e durante il fine settimana

 

Produzione e datacenter:

  • Abbiamo iniziato a proporre ai nostri clienti un'infrastruttura sostitutiva (server dedicati, Cloud privato e pubblico) nei datacenter di Roubaix e Gravelines
  • Abbiamo previsto di installare catene di montaggio supplementari nel corso delle prossime 48 ore per triplicare la nostra capacità di produzione

 

Sul sito di Strasburgo:

  • Diagnostica e inventario in corso nella giornata di oggi
  • Pulizia e riparazione degli edifici danneggiati in corso per garantire un ambiente di sicurezza per i nostri team
  • La pulizia della schiuma e l'aspirazione dell'acqua sulla strada,  realizzate. Oggi è prevista l'aspirazione del serbatoio d'acqua.
  • SBG2 deve essere ricostruito quasi completamente.
  • SBG1 è stata gravemente danneggiato.
    • Sala di rete SBG1 - Sala intatta in seguito alla verifica preliminare. ETA: inizio della settimana prossima.
    • ETA provvisorio per l’elettricità = lunedì 15 marzo
    • Nei giorni seguenti i server saranno attivati pezzo per pezzo dopo l’audit
  • Per SBG4: le verifiche preliminari e non rivelano alcun problema. L'obiettivo è ora quello di ripristinare l'elettricità durante la settimana 12 e di ripristinare gradualmente tutti i servizi.
  • SBG3 non è stato colpito dall'incendio. L'obiettivo è ripristinare la corrente e la rete durante la settimana 12 e ripristinare gradualmente tutti i servizi

Le nostre tre priorità sono:

  1. Riservare infrastrutture sugli altri datacenter per i clienti coinvolti: nei siti di Roubaix e Gravelines, abbiamo a disposizione un inventario di nuovi server pronti alla consegna presso la maggior parte dei clienti coinvolti. Nelle prossime settimane aumenteremo ulteriormente la disponibilità nei datacenter con la produzione di circa 10.000 nuovi server. La procedura sarà comunicata quanto prima ai clienti interessati.
     
  2. Proteggere il sito, al quale abbiamo accesso, pulirlo e ristabilire l'elettricità e la rete per i tre datacenter coinvolti.
     
  3. Continuare a valutare con i nostri clienti l'impatto sui loro servizi localizzati nei datacenter interessati per trovare le migliori soluzioni.
     

Azioni aggiuntive intraprese da OVHcloud:

Rete:

  • Tutte le connessioni in fibra che entrano nel Datacenter funzionano correttamente.
  • Tutti gli ordini sono stati effettuati presso vari fornitori. ETA: in loco l'11 marzo.

Comunicazione:

Il nostro compito è fornire ai nostri clienti la più alta qualità di servizio per sostenere le loro attività online e sappiamo quanto sia importante per loro. Ci scusiamo per i problemi causati dall’incidente. Ci impegniamo a continuare a comunicare con la massima trasparenza sulla causa dell’incendio e sulle sue conseguenze.

Un'inchiesta è attualmente in corso. Indipendentemente dalla causa dell'incidente, OVHcloud sta valutando le potenziali misure tecniche e soluzioni operative da fornire ai clienti coinvolti.

Tutti i nostri canali di comunicazione, a partire dalla piattaforma di monitoraggio degli incidenti travaux.ovh.com, incluso il task), sono accessibili e aggiornati in tempo reale.

Le FAQ sono disponibili qui.

Contatto Stampa: press@ovhcloud.com

------------

10/03/2021 - 21:00 (CET)

Riepilogo dei fatti:

  • Un incendio si è verificato mercoledì 10 marzo 2021 alle 00:47 in una sala di uno dei 4 datacenter OVHcloud situati a Strasburgo, chiamata SBG2.
  • L’incendio è stato circoscritto all’inizio della mattinata.
  • Non vi sono feriti.
  • L'incendio ha distrutto principalmente il datacenter SBG2 e ha parzialmente danneggiato il datacenter SBG1 (4 sale distrutte su 12). Gli altri due datacenter OVHcloud di Strasburgo non sono stati coinvolti nell'incendio; i server SBG3 e SBG4 sono in fase di disattivazione ma non sono stati interessati dall'incendio.
  • Precisiamo che il sito non è classificato come Seveso. 
  • L’origine dell’incendio è attualmente in fase di accertamento ed è stata avviata una procedura d’indagine da parte delle autorità.

Ecco le azioni intraprese da OVHcloud:

  • I team tecnici e commerciali sono mobilitati da stamattina per informare i nostri clienti e rimediare all'indisponibilità del nostro sito di Strasburgo.
  • Octave Klaba, fondatore dell'azienda, è sul posto da stamattina con i team industriali e tecnici.
  • Per seguire in tempo reale l'evoluzione della situazione:
  • Task travaux: http://travaux.ovh.net/?do=details&id=49484
  • Twitter: https://twitter.com/OVHcloud_FR
  • Una comunicazione per i clienti e una FAQ sarà disponibile a partire da domani.
  • Per facilitare la gestione delle richieste dei clienti, consigliamo loro di creare ticket sul nostro sito o consultare il nostro centro assistenza.

Le nostre tre priorità sono:

  1. Riservare infrastrutture sugli altri datacenter per i clienti coinvolti: nei siti di Roubaix e Gravelines, abbiamo a disposizione un inventario di nuovi server pronti alla consegna presso la maggior parte dei clienti coinvolti. Nelle prossime settimane aumenteremo ulteriormente la disponibilità nei datacenter con la produzione di circa 10.000 nuovi server. La procedura sarà comunicata quanto prima ai clienti interessati.
  2. Proteggere il sito, al quale abbiamo accesso, pulirlo e ristabilire l'elettricità e la rete per i tre datacenter coinvolti.
  3. Continuare a valutare con i nostri clienti l'impatto sui loro servizi localizzati nei datacenter interessati per trovare le migliori soluzioni.

OVHcloud si impegna al massimo per garantire la continuità dei servizi ai propri clienti:

  • Stiamo lavorando a un piano per riavviare i tre datacenter non interessati (SBG3 e SBG4) o parzialmente interessati (SBG1) e la nostra rete il più rapidamente possibile.
  • Ti invitiamo a prestare la massima attenzione alle email ricevute: in un periodo di crisi, i tentativi di malware (phishing, spam, sollecitazione, ecc.) si moltiplicano. E' più che mai necessario restare vigili. 

L'impatto sulle nostre attività:

  • Continuiamo a valutare l'impatto dell'incidente, in particolare per i clienti i cui dati si trovavano nel nostro datacenter distrutto dall'incendio.
  • I nostri servizi Web VOIP in Francia non sono stati coinvolti.
  • Tutti i nostri servizi negli altri datacenter in Francia e nel mondo (di cui 15 DC in Europa) sono pienamente operativi.

Il nostro compito adesso è di offrire ai nostri clienti una qualità di servizio ottimale per continuare a garantire le attività online, pienamente consapevoli di quanto queste siano importanti. Ci scusiamo per le difficoltà che questo incendio ha causato, e ci impegniamo a comunicare con la massima trasparenza le cause che lo hanno generato e le relative conseguenze. Stiamo valutando l'impatto dell’incidente e comunicheremo quanto prima, con la massima trasparenza, lo stato di avanzamento delle analisi e l'attuazione delle soluzioni.

Stiamo inoltre valutando l’impatto ambientale in collaborazione con le autorità competenti e seguendo una procedura per confermare l’assenza di inquinamento.

In questa fase, possiamo confermare che i residenti non corrono alcun rischio.

Valutiamo costantemente l'impatto di questo incidente e forniremo, quanto prima e con la massima trasparenza, una comunicazione sullo stato di avanzamento delle nostre analisi e sull'attuazione delle soluzioni.

Tutti i nostri canali di comunicazione, a partire dalla piattaforma di monitoraggio degli incidenti travaux.ovh.com, incluso il task), sono accessibili e aggiornati in tempo reale.

Task travaux: http://travaux.ovh.net/?do=details&id=49484

------------------

10/03/2021 - 11:00 (CET)

Mercoledì 10 marzo 2021, alle 00:47, si è verificato un incendio in una sala di uno dei nostri 4 datacenter di Strasburgo, SBG2. Precisiamo che il sito non è classificato come Seveso.


I vigili del fuoco sono immediatamente intervenuti sul posto per proteggere le squadre di intervento e limitare l'aumento dell'incendio. A partire dalle ore 2.54, il sito e il suo perimetro sono stati completamente isolati. Alle 4:09, il fuoco ha distrutto SBG2 e continuava a presentare rischi per i datacenter vicini fino a quando i pompieri non hanno preso il controllo completo dell'incendio. A partire dalle 5:30, il sito è stato interdetto ai nostri team per ovvie ragioni di sicurezza e sotto la guida della prefettura, l’incendio è stato ufficialmente limitato. Siamo lieti che non ci siano feriti, né tra i nostri team né tra i vigili del fuoco e tra il personale della prefettura, che ringraziamo per l’esemplare mobilitazione al nostro fianco.   


Grazie al parco operativo di 15 datacenter in tutta Europa, i nostri team tecnici e commerciali sono totalmente dedicati ai clienti per implementare soluzioni e rimediare all'indisponibilità del sito di Strasburgo.   Il nostro compito adesso è di offrire ai nostri clienti una qualità di servizio ottimale per continuare a garantire le attività online, pienamente consapevoli di quanto queste siano importanti. Ci scusiamo per le difficoltà che questo incendio ha causato, e ci impegniamo a comunicare con la massima trasparenza le cause che lo hanno generato e le relative conseguenze.

Stiamo valutando l'impatto dell’incidente e comunicheremo quanto prima, con la massima trasparenza, lo stato di avanzamento delle analisi e l'attuazione delle soluzioni.


Tutti i nostri canali di comunicazione, a partire dalla piattaforma di monitoraggio degli incidenti travaux.ovh.com, incluso il task), sono accessibili e aggiornati in tempo reale.

Service PresseOVHcloud

press@ovhcloud.com

Il tuo account clienteContatto commercialeWebmail OVHcloud Blog

Benvenuto in OVHcloud!

Accedi per ordinare, gestire prodotti e servizi e monitorare gli ordini

Accedi