Disaster Recovery: A New Era of Cloud-based Resilience

The unexpected global events of the last few years, coupled with the ascendancy of cloud-based operations has furnished the ideal conditions for broadcast disaster recovery (DR) and business continuity (BC) to take a decisive step forward.

At least that’s the firm impression yielded from a few days of speaking with broadcast consultants, vendors and cloud specialists, who indicate an increased desire by broadcasters to prepare for the worst possible scenarios – and ensure that their operations do not suffer unduly as a result.

“In the way that security has a much higher profile now than it used to, the same can be said for DR and BC for many of our customers,” said Andy Warman, Senior Vice-President, Product, Imagine Communications. The company recently launched its Aviator Orchestrator platform for synchronisation across on-premise sites and in the cloud.

“The trigger to take action seems to be the value of the assets, which is not too surprising given [their importance] to broadcasters, content owners and so on.”

While many will already have strong “procedures and practices in place, [it’s vital to show] they are not standing still and the way they think about these issues is evolving.”

Read more Putting a price on disaster

It’s not difficult to see why the geo-dispersed, zonal potential of the cloud – in which the unavailability of assets in one data centre can be seamlessly addressed by their retrieval from another – is resonating so much with broadcasters that were already in the process of embracing more distributed working patterns.

Disaster Recovery: Costs have ‘dropped considerably’

Things might be moving quickly in the DR/BC space now, but it’s certainly been a long time coming, as Craig Bury, CTO of media consultancy Three Media, can attest.

“When I first started trying to do more DR work around 1998, everybody had their own data centres,” he recalled. “To build a DR or even BC facility was expensive and nobody wanted to do it, really. Now with so many services in the cloud and more people moving there every day, the ability to implement BC is actually very straightforward. [For example] if you look at the use of automation tools like Kubernetes, Rancher, Docker and Swarm – in combination with something like Terraform [open-source software] – it’s possible to provide almost purely automated recovery, and not only in the area where you are located [thanks to multiple availability zones].”

As a result of which, the costs of deployment have “come down by multiple orders of magnitude,” observed Bury. But if it’s undoubtedly become easier – and cheaper – to implement effective BC/DR due to recent advances, another question still remains: Why now?

Disaster Recovery: Resilience ‘baked in’

Most observers agree that multiple factors have contributed to the recent upsurge of interest in DR & BC. Confirming that they have become “higher priorities” in recent years, Baskar Subramanian – co-founder and CEO of cloud-based SaaS technology company Amagi – alluded to critical workflow and cyber-security concerns.

“One key driver behind this trend is increasing reliance on technology to produce, manage and distribute media content,” he said. “This dependence on technology means that any disruption to these systems could result in significant downtime and financial losses. Another driver is the rising threat of cyberattacks and other security breaches that can compromise sensitive information, disrupt operations, and damage a company’s reputation.”

Simon Eldridge, Chief Product Officer of media supply chain management firm SDVI, pointed to “some fairly high-profile cases where people have lost the contents of their private data centres or had fires or other issues.” But the arrival of the pandemic heightened the awareness of “what a cloud-based system meant in terms of distributed access to a distributed workforce; the whole thing became more interesting than it previously had been.”

Subsequently, further investigation by broadcasters tended to elicit the realisation that cloud-based DR could remove a whole sphere of potential pain-points, including back-up issues and environmental factors. In the past, broadcasters “may have been approaching DR in terms of buying two of everything, [eg] a primary system and a back-up one that is only used in emergencies… But if the solutions they are putting in place now are truly cloud-native, they will have DR and resilience ‘baked in’.”

Disaster Recovery: Priorities and tiers

If the cloud is now overwhelmingly providing the basis for fresh DR/BC deployments, it’s unsurprising that underlying economic (value of content) and practical (how content is to be accessed and managed) considerations tend to determine the precise configurations for each customer.

This is confirmed by the quick checklist offered by Warman when asked how companies should approach reviewing their DR/BC capabilities. “Firstly, they need to think about the value of the content and the perceived value of the channel,” he said. “So what it means to you and what the loss of service means to your customers and your advertising revenue. Secondly, they need to review the economics of standing up proper DR. For persistent channels that have been on-air for years, it makes sense to have a replica as there is an established high value and it’s easy to justify. But for a new or niche channel where the economics are not yet known or are changing, a more elastic style of DR [may be more appropriate].

“Finally, it’s important to understand how you want to look at and manage your assets. You have to be clear on where the assets can be kept to ensure you are able to deliver if you need to do something in a pinch.”

For SDVI, Eldridge said that customers’ approach to DR tends to be separated into four tiers of increasing importance, beginning with “a service that is important but not mission-critical, and if it’s not available it won’t affect the main channels and D2C offerings; the next tier up is where you want to have some distributed processing of the media, which means you can spin up the processing wherever the content is and then move to a second region [if the first one] goes down; the next tier up is multi-availability zones involving separate data centres or separate sections of a data centre; and then the top tier is complete, multi-regional backups – for example, that might include sites on the east and west coast [of the US], or Frankfurt and Dublin.”

Disaster Recovery: View through a ‘geopolitical prism’

Given the proliferation of risk factors – both existing and emerging – it is surely just as well that cloud-based DR & BC have attained their present level of maturity. In particular, with climate change already bringing a huge increase in the number of extreme weather events, and the world surely at its most unstable since the end of World War II, broadcasters are increasingly obliged to perceive their DR capacities through a geopolitical prism.

For Amagi – most of whose partnered cloud providers offer DR as a service (DRaaS) –Subramanian remarked: “The threat of sustained attacks as part of wider geopolitical events is a particular concern as these attacks can be more sophisticated, targeted and persistent than typical cyber threats. [Meanwhile] climate change is expected to increase the frequency and severity of natural disasters. More frequent disasters increase the probability and recurrence of disruptions, while more severe disasters increase the length of time before those disruptions are resolved. Broadcasters may need to revisit their disaster planning and preparedness strategies to ensure they are ready for more dramatic weather-driven events, more often.”

Last word to Bury, who remarked that the new approach to DR and the rise of hybrid & distributed working can be seen as “two sides of the same coin, requiring a lot of the same skills. One of the great things the cloud gives you is the opportunity to review the way you work on a day-to-basis, and the resilience and redundancy that you need. If you are carrying the World Cup, then you need total resilience, but it may be different with other channels. With the cloud you have the chance to think about that very carefully and then tailor your operations accordingly.”

Read more On-prem vs Cloud Storage: Towards a Hybrid Future=