As the industry turns to remote production solutions during the coronavirus, independent post house The Farm is taking the opportunity to accelerate its own remote post production plans.
For a lot of the content industry the remote workflows that are suddenly being imposed by necessity are either proving a rude shock or being dismissed as simply unworkable. But a fair section of the post production sector it has merely been an acceleration of programs that have already been in development for several years.
Driven in turn by exorbitant rents in city centre locations, the growing desire to pursue a saner work/life balance, and the need to take advantage of the economic benefits that cloud workflows can bestow, many facilities have invested a significant amount of time and energy into setting up remote workflows.
The Farm is one of them, and its reaction to the Covid-19 outbreak is a good example of what can be done at speed once the initial groundwork has been established.
“Over the last year or so we’ve been working at ways of connecting people collaboratively and finding ways for them to work with our infrastructure,” says operations director Jack Edney. “In the coming months we were about to start working out some new remote working solutions anyway, so we’ve already been prepped to a great extent which has been beneficial, but this obviously has made everything slightly more urgent.”
The company’s list of its current remote operations is an impressive one and growing on a weekly basis in a carefully managed roll out. 70 remote editorial systems have been set up running off The Farm’s central storage via the Teradici PCoIP, there are 22 managed offline suites, 40 staff operators, 15 online editors, 11 ProTools audio suites (three of them working in 5.1), and one Nucoda grading suite which has been set up as a test bed to see what the issues involving rolling out remote grading to the home might be.
All Farm locations in London, Bristol, and Manchester are hooked into the distributed network, and shows currently going through it include Britain’s Got Talent (Talkback Thames for ITV), and Hospital (Label 1 for BBC2). It was a gradual roll out, with the company taking about 10 days to let any non-Farm employees inside the remote offline system, and the bringing them on board 10 at a time.
“We started with editorial and offline editing,” explains technical director Adam Peat. “What we’ve done there is supply access remotely using the Teradici system, which simplifies connecting in so there are no tricky VPNs for non-Farm staff to have to use; most of the people on that are freelance editors and producers.
“That gives us a gateway in directly through our internet connection to a connection broker that then ties users into their relevant edit systems. They install the software, point that at our gateway in, log in with the username and password we give them, and that’s done.”
Farm staff, in both admin and operations roles such as edit assistants, have been set up using HP’s Remote Graphics System (RGS), which is very similar to Teradici but allows for the fact that they are already part of The Farm ecosystem and have the required VPN access. An editor using that will log onto a Flame or a Symphony and the edit assistant staff will be logging into multiple systems - whatever they need to do to do their tasks.
Alongside such KVM work, in parallel The Farm has been working on streaming.
“We have various options for different use cases,” explains IT director, Adam Morris. “We have the editorial use case where low latency is important but it’s not the be all and end all. We’re currently trying to get that into browser format, but low latency in browser players is fairly proprietary at the moment; there’s some new stuff that Apple and MPEG have done but support is varied.
“The problem we’re trying to crack at the moment is that near realtime collaboration between producer and editor, whether that editor is in The Farm or sitting in their pants in their front room,”
“The other use case where the picture needs to be good and latency relatively low includes a realtime collaborative sign-off process from an online suite. And then at the top of the food chain you have colour accurate remote, view and approve. Doing that at anything over 10bit has been a struggle in terms of getting the output out of the facility and then getting it seen at the other end. There are various things you can do at 10bit that are quite good and quite easy, but as soon as you go above that you are in a whole new world where you need hardware decoding ideally. It’s all possible but it’s problematic.”
It’s a demanding system and time is of the essence in setting it up. In its favour The Farm already has its own portal, FRED, which it uses for remote viewing and approval, but all the same trying to hook between 150 and 200 people a day into a genuinely distributed workflow is a task best described as non-trivial.
“I think that the challenge for us is the scale that we’re rolling out,” says Edney. “It’s fairly easy to roll out one system with a couple of screens and even a video monitor off the back of it. But trying to find products that will fit into our ecosystem and scale very well, that’s the biggest challenge.”
“The problem we’re trying to crack at the moment is that near realtime collaboration between producer and editor, whether that editor is in The Farm or sitting in their pants in their front room,” adds Morris. “There’s a fourth thing we’re looking at too which is getting a second or third screen, your traditional second or third screen monitoring, and being able to deliver that to people’s front rooms via Teradici or another system.”
Connectivity challenges
There are, of course, multiple issues when setting something like this up, not least of which is bandwidth. Even in areas of the country with ultrafast fibre, supporting the 1.5GBps that 16bit workflows within a facility demand is going to be impossible, which is why The Farm considers in many cases remote collaboration to be stage one of a two-step process.
“Our methodology on it is, is that we will enable people to work remotely, and they can do as much work in a compromised scenario as they need, but a lot of it will be prep work so when it comes to the high end finishing they’ll then need to come into a facility,” says Edney.
“Even if they’ve done some form of a proxy workflow, they need to come in and have a watch back and make sure the content is exactly as it should be. But as long as they’re comfortable or safe getting to and from work, once they’re within our buildings they’re really isolated anyway because they keep to the same room.”
Content security in all this is something of an elephant in the room question. As Edney points out, The Farm can control everything within its physical perimeter, but once content ventures outside of that it is, by nature, insecure. Multi-factor authentication (MFA) is enabled across The Farm’s VPN, and a MFA-enabled version of Teradici is currently undergoing testing, but TPN best practice demands that nobody in the content chain should be on a device that can potentially take a screenshot, but that is a very difficult thing to police in somebody else’s home.
“There are a whole raft of issues that the industry needs to get too grips with fairly quickly,” he says on the subject, adding that despite the high level of security already maintained there needs to be more investigation of technologies such as live watermarking.
Feedback, so far, has been good - “I’ve actually been quite surprised by how content people have been with it,” says Edney - and The Farm reckons that it now has a remote offering that covers around 80% of current scenarios. With a lot of that being bespoke development as well rather than off-the-shelf solutions, that means that when the industry comes out of the other side of this there is a solid, viable product that has been developed as a result.
“The journey is a challenging one, but also at the same time it’s the best R&D playground any technologist could ever have right now because you can do anything you like within boundaries,” says Edney. “What it’s done is that it’s accelerated what was going to happen over the course of a few years to happen over the course of a few months.
“I would like to think that at the end of this there will be many productions who will be content with a new way of working and embrace it as a result. And that could be really beneficial for the post production sector longer term because it will be able to minimise some of the extortionate costs that come along with having thousands of square foot of real estate rent and rates.”
- Read more: Halo post production goes virtual
1 Readers' comment