How We Built It: PFT offers Clear vision for C4 cloud control

From its new home in Leeds, Channel 4 is taking advantage of a new hybrid-cloud media centre, capable of managing all media processing, as well as centralised content operations including live simulcast and VOD services.

The facility has been built by Prime Focus Technologies (PFT) and is based around its media-focused enterprise resource planning (ERP) software, CLEAR.

Cloud-based workflows will provide a scalable infrastructure for Channel 4’s operations, including receipt and delivery of content, quality control, edit and remastering, archiving, live simulcast of linear channels and transcoding, along with packaging and delivery of non-linear services. The end-to-end content supply chain is managed through CLEAR’s SLA-driven media processing services platform,

C4 Leeds the way

In 2019 Channel 4 announced it was to move its national headquarters from London into the Leeds area as part of the nations and regions strategy.

Previous to C4’s move north, all services had been provided by one vendor, Red Bee Media for the past ten years.

“Channel 4 decided to split the outsourcing, breaking it up into effectively five services; twelve vendors pitched for it,” says Nick Kaimakami, executive vice president and head EMEA and USA East at Prime Focus Technologies.

“So there was media services, the online services, the playout, the access services, and the post-production, which would have previously been done altogether by one vendor. We were invited to bid for three of those, and won the media services and the online services.”

Channel 4 uses a legacy system to manage all of its content. “It’s like the heart of Channel 4, it’s what keeps everything going,” says Kaimakami. “In terms of technology, what Channel 4 wanted was a platform that they can take into the future. Innovation was a key driver - [they wanted] something that gave them very good dashboards, great visibility of what’s going on, rather than having an outsourced black box. Now by having access to CLEAR, they can see what’s going on.”

CLEAR now provides MAM and workflow orchestration for all the media services for all of Channel 4’s content. The full end-to-end services offered include QC services and QC fixing, remastering of acquisitions and re-purposing content, archive services, compliance and re-versioning services, digital distribution services and access to PFT’s global media services pipeline.

“Normally we will drive the services element from offshore because our CLEAR platform allows us to do that anywhere, but Channel 4 wanted to keep jobs in the UK and we were happy to create jobs here,” says Kaimakami. “So we said, we will deliver it from wherever you are. We selected the Leeds location as somewhere we could expand and be close to the facility; it’s about 10-15 minutes walk from the Channel 4 headquarters.”

“There were about 8 petabytes of content, that was sitting with the previous vendor, which we had to then migrate across,” he continues. “That consisted of all existing content and archives plus the ongoing material that continued to go out during that process, so we had to set up a facility to get the migration going and move all the content across.

”Cloud is the right way to go, it offers you flexibility and you can scale up,” Nick Kaimakami, Prime Focus Technologies

“The way the architecture works is that we will keep everything on-site, so we’ve got a new Spectra Logic tape library that holds everything that’s come over from the previous vendor,” he adds. “We also keep a copy in AWS, and in case of a disaster recovery we can pick it up from our London office or Mumbai and we can move people around and work remotely.”

Building blocks

The location of the physical facility, PFT Leeds, was chosen soon after signing the strategic deal with C4 in September 2019.

“We chose Leeds because the council started to pull some funds together to create more studios, to attract more media into the area,” Kaimakami says.

“We’re surrounded by Leeds Studios, which is part-owned by the council, and anybody can come and do some filming. Then potentially they can pop into our facility next door and we can take the content and then it can go anywhere.

PFT spent 2020 building out the infrastructure, commissioning, testing and hiring for the new stat-of-the-art facility.

”The planning, design, software and operations stages however all had to be done remotely pretty much through Covid. In a very large and complex programme like this where there are many parties and many moving parts you ideally want to have everyone in the same place, and whilst we started that way, once Covid hit, it naturally slowed everyone down.

“Our software development is done out of India, and the teams out there have done a fantastic job in very difficult circumstances for the last couple of years, particularly the last year in coping with [the pandemic],” he adds.

“The first wave of people, the first super users, arrived in September, to perform systems integration tests” he adds. “We’ve been ramping up the staffing, all the way through to this coming August when the last wave arrives.”

Access to the cloud

“We have the data centre server in Leeds housing our archive, near line storage and some heavy duty compute, but actually 90% of the activities are in the cloud,” says Andy Steele, vice president of operations for Prime Focus Technologies.

“Our CLEAR platform is effectively the heart of Channel 4 operations now, with day to day prioritisation and the kind of live status updates [that didn’t exist before].

“CLEAR is very tightly integrated with Channel 4’s BMS system which sends us work orders for the addition of subtitles or a programme being allocated to have access services,” he adds. “CLEAR will interpret that and apply business rules and set all the due dates appropriately, right the way through the workflow. It will then integrate with the playout vendor’s access services and [report back into the system] to give live status updates.

“The headline around that is no more need for a Google Doc to prioritise the key scene that needs to be done for tonight’s Gogglebox,” he continues. “Instead all of that data is handled programmatically and the status updates between all the players are exchanged automatically over APIs. So everybody knows exactly where everything is. If Channel 4 then change their schedule that ripples through CLEAR and on into access services, for example.”

“CLEAR is also automating all the ingest of content that comes in from 300+ independent production companies that are commissioned by Channel 4,” Kaimakami says. “It’s doing all of the quality checks and preparation that is needed to make it ready to go to access services and then brings it back to be played out. Then of course there’s the reporting and a lot of additional features that CLEAR as a platform offers Channel 4.”

Kaimakami points out that like every other broadcaster, C4 has a big focus on digital.

From an online perspective, we use CLEAR to manage everything around the preparation of content going on to the digital platform [All 4], as well as all the monitoring once it has aired [to check] the quality of service.”

“We bring together a number of cloud toolsets under the CLEAR suite, which all happen rather seamlessly,” says Steele. “Say for example Channel 4 decides it wants box sets of the West Wing to suddenly appear on All 4, they will go into their system and schedule it and a work order comes across with a bunch of data. We’re hands-off in Leeds, as it’s all automated: the transcoding, all the renditions, the packaging, the delivery, and all the acknowledgements of the API’s inbound and outbound with the third-party platforms and back with Channel 4’s BMS system; that’s all done entirely in an automated way.

“We’ve got some very good monitoring tools, which tell us and the support team in India when something’s not working. That’s also automated,” he adds.

“The big innovation for Channel 4 is they can see on our dashboards exactly where everything is at any one time. All of that manual thinking has now been taken away and turned into business rules within CLEAR. You don’t need to be living in email all the time anymore, in this login screen you can see exactly where your stuff is and search for it.”

“The reality is that whether it’s in AWS, or actually in Leeds, it is invisible to the operator,” he adds. “To anyone logging in, you can’t tell where [the operation] is, you don’t need to know.”

Ready to serve

Now fully operational, in line with UK Government guidelines on COVID-19, the building also now forms the headquarters of PFT in the UK, after it closed one of its two offices in London. Eventually it will have a staff of 70.

“Anything that needs operational delivery work from the UK will be done out of Leeds,” Kaimakami says.

“We’ve designed PFT Leeds to be a hybrid on purpose, to achieve the rich variety of handling of content inbound and outbound,” he adds. “We still have a server room in our operation in Leeds, where we house the few things that still need to be done on-prem. This is mainly [to support] legacy formats. Some third-party platforms in the UK require a very specific and non-standard output and therefore some toolsets still have to be quite specific and can only be hosted in the data centre.”

But he adds: “Cloud is the right way to go, it offers you flexibility and you can scale up, which is another thing which Channel 4 wants us to do. C4 is already the UK’s biggest free streaming service. They’ve got big plans for their digital future to follow that strategy. We are aligned with that and set up to support it with what we’ve built.”