Google's vision for machine learning at "every step" of the production process

Speaking at last week’s annual SMPTE 2017 Annual Technical Conference & Exhibition, Kember set out the latest advances in ML and described how artificial intelligence (AI) and ML could shape the future of the media and entertainment industry.

Kember, who previously worked as a CG Supervisor at animation and VFX facilities Tippet Studio and Framestore, described ML applications in media as an area “near and dear to my heart.”

“Machine learning at every step of the way is where we are going with this” - Jeff Kember

Kember said: “I jumped out of production because I wanted to change how the industry worked, but I couldn’t do it from the inside.”

According to Kember, Google is “taking the entire production pipeline for episodic TV or features and making it available globally,” from the entire media workflow from camera footage to archiving and disaster recovery.

He said: “Instead of saying ‘let’s get all the way to the end of the pipeline, where the image is complete and ready for distribution’, we wanted to look at how to insert ML in the entire production cycle; I wanted to take raw footage from every camera and use something like Colorfront to debayer it, run a machine learning pass to generate metadata and JSON throw it into a MAM, do a transcode and produce dailies. That allows you to decentralise your editorial process; you can shoot in a field and have editorial staff elsewhere in the world.

“This provides the ability to get anything back in a couple of milliseconds, and the ability as well to have global access to it.

“We want to provide a framework where you can hire any people you want globally to do this kind of work, and we are working to build in machine learning at each and every step along the way to provide not only image analysis but also automatic scene breakout, and provide an efficient way to generate an EDL and transcode into the correct flavours based on what might have been used before.

“Machine learning at every step of the way is where we are going with this.”

However, Kember acknowledged the hype currently surrounding AI and ML and cautioned against the belief that AI and ML would solve all of the industry’s problems.

“[Google is] taking a slightly more measured approach to it and trying to identify problems that need to be solved and to look at where it could go in the future.

“For us, ML is a branch of artificial intelligence, but it is also an enablement to create machines that improve themselves over time. We like the idea that it is an opportunity to solve problems without having to specifically codify the solution.

Kember also revealed the extent to which deep learning has been adopted by Google; a slide during his keynote illustrated a steep increase in the use of deep learning directories across Google’s product lines, with some 5,000 in use across the firm’s apps including Google Search, Android, Maps, Play, Gmail and YouTube.

Finally, Kember suggested that the impact of AI could be as profound and far-reaching as the effect that the steam engine had on industry and society.

“If the industrial revolution saved us from physical drudgery, how do we get past mental drudgery? That is where we see ML come in, with the ability to empower humans and move forward the technology and capabilities we have, and to do so in a shorter amount of time.”

• The annual SMPTE 2017 Annual Technical Conference & Exhibition took place October 23 to 26 at the Hollywood & Highland Center in Los Angeles.