The week after

It was the 10th of March when not only we but also you stood before a heap of ruins.

The tragic incident in Strasbourg continues to preoccupy us to this day: https://www.heise.de/news/OVH-Feuer-zerstoert-Rechenzentrum-in-Strassburg-ein-weiteres-beschaedigt-5076320.html

Strasbourg has always been our main location since 2013, when it all began, and accordingly many environments lay idle from one day to the next.

Very quickly we got the findings that the inventory of the backups was unsafe and we would have to wait for them. Until today we are waiting for the arrival of these backups. Backups which, mind you, were paid for in 2 separate environments and were guaranteed by a contractual assurance of 100% on the part of the data center.

In general, our plan always provides for data redundancy. This data distribution has always extended to Strasbourg 1, 2, 3 or 4. We had not even begun to plan for a day when all 4 data centers would be idle.

In the future, such scenarios will also be incorporated into our processes.

What is the current status?

All websites and web radios are up and running again. Especially for larger radios with user numbers of 500 listeners upwards, an outage also means loss of revenue. Within 3 days we delivered replacement systems to 200 webradios that were affected by the cloud outage.

But we still have a lot of work to do here, many of the stations run on spare systems which were / are not actually intended for the scenario in question.

On the part of the users there was, despite everything, a lot of understanding and little criticism of what was happening. We thank all operators of every radio size for the continuous understanding!

Which services are still available?

The following STREAMPANEL environments are currently available

• Time Accurate Content
• Safeguard
• content syndications
• Web Radio Apps
• STREAMPANEL Free (The service is completely destroyed by the fire and until everything runs again orderly stopped! The problem is in the current situation, the cost factor!)

Here we are dependent on instance or file-based backups. If both components are no longer available, the further procedure is still open.

In some environments which work in principle, there are sometimes delays in the loading time. This is due to missing dependencies.

What's next?

We are closing the platform for the time being and no longer allow new customer orders. Existing customers can contact us via the ticket system. Automatic upgrades and immediate adjustments to current conditions are currently suspended. The autonomy of the platform is currently not given.

Still we are quasi blind. but that changes again. In the meantime it is called. Tell us everything you notice!

We currently think this is the best course of action and don't want to create an impression of chaos with new users 🙂 .

THANK YOU!

You can find the status history of the incident here.