The holodeck emerges into the light

The Abbatars of pop giants ABBA begin their Voyage tour in a purpose-built London arena this May, but don’t be fooled into thinking these are holograms.  

The expertise of over 100 digital artists and technicians from ILM promises to make the experience the most sophisticated rendition yet of Peppers Ghost, a magic trick popularised in the Victorian era. But it remains a 3D projection of a 2D source lacking parallax or the ability to look around the 3D image. 

This is the breakthrough that US firm Light Field Lab claims it is nearer to producing than any other company. It has already begun manufacturing the technology to sold out pre-orders which will be installed in commercial environments from next year. 

“We don’t defy any laws of physics. Perhaps it may seem like we are but there’s some neat optical and electrical science that backs up exactly what we are achieving,” Jon Karafin, Light Field Lab

“Over the next few years you will see the release of larger and larger holographic walls,” says CEO Jon Karafin. “From there you can create a very large holographic volume ultimately leading into our vision for the holodeck where there’s no limitation to where the objects can exist in 3D space.” 

To do this, Light Field Lab has developed technology capable of emitting particles of light called photons from a custom display that is not only able to generate light from billions of pixels but is able to converge the light in waves at a specific point in space rendering a three-dimensional object visible to the naked eye. 

“One of the most important rules is that you can’t freeze a photon in mid-air,” Karafin adds. “You are either projecting onto a medium or it is VFX. We don’t defy any laws of physics. Perhaps it may seem like we are but there’s some neat optical and electrical science that backs up exactly what we are achieving.” 

Behind the technology 

The key to creating a holographic moving image, according to Karafin, is pixels. Lots of them. 

“It is the pixels that allow us to form objects in mid-air,” he says. “Pixels dwarf the other cost drivers, including electronics, controllers, computation and optics. To date, the inability to create a large digital display of sufficient resolution to form holographic objects has limited the proliferation of real digital holograms. 

“An accurate recreation of the behaviour of light in three-dimensional space allows viewers to see full parallax from any angle and displays an image that’s indistinguishable from the real thing. But it also requires an increase in resolution of multiple orders of magnitude compared to a traditional flat display.” 

Light Field Lab’s basic technology, branded SolidLight, is a 28in panel that is able to output 2.5 billion pixels, or more than 10 billion pixels per square metre.  

“It is doing this at full P3 colour space in a way that has not been possible until now,” he says. “Just one of these panels by itself is already the highest resolution display that has ever been designed.” 

Each panel packs a proprietary FPGA, GPUs, wall and display controllers. In front of that is the key innovation of a ‘nano particle polymer surface energy relay’. “This gives us the opportunity to form a singular continuous wavefront composed of cones of light,” he explains.  

The wavefront contains amplitude based on patterns and frequencies that is being modulated by a phase guide that sits on top of the polymer surface.  

“Just one of these panels by itself is already the highest resolution display that has ever been designed,” Jon Karafin, Light Field Lab 

“The phase guide control plane is what allows us to converge the amplitude into a focal region,” he says. “In order for anything to exist as a true wavefront you need extremely high density sampling to generate billions and billions of photons.” 

Since it is modular and bezel-less, these panels can be combined to form video walls of any size with configurations exceeding hundreds of billions of pixels. 

“You can go from forming an object the size of a single seahorse to a whole aquarium of fish and dolphins. As you increase your photonic budget you are able to do more things – meaning you can increase your angles or projection depth. You can trade off these different parameters depending on what application you need.” 

• Read more Understanding the metaverse

For example, the standard viewing angle is 60-degrees but this can range up to 120-degrees. By making the objects smaller the viewing angles can be wider. 

Light Field Lab is initially targeting “premium video wall installations” such as corporate lobbies, out of home advertising sites, museum installations – all markets where a wow factor is important and where many customers already have media servers to drive the content.  

Video wall applications 

Light Field Lab’s own projections for the market for larger video walls show growth from about $19 billion today to nearly $26 billion in five years. The fastest growing segment is for high-end panels with narrow pixel pitch – not coincidentally, the market segment where holographic offerings will play. 

IMARC Group expects a growth rate of around 12% to propel the video wall market to $32.1 billion by 2025, and Fortune Business Insights sees the market reaching $36.16 billion by 2026. Both forecasts make Light Field Lab’s own look conservative. 

Its kit is being sold turnkey, including the software and plug-ins required for customers to generate holographic experiences. 

A WaveTracer, for example, plugs into any existing third-party content creation package like Maya, Unreal, Unity. “You don’t have to do anything differently,” Karafin says. “You could use the content you’ve already generated and then in real time you can see that running on the holographic display.” 

Any content that has a depth pixel associated with it can be displayed. That includes 3D movies. Since all Disney’s MCU films are processed this way does that mean you could playback Spider-man: Far From Home on a SolidLight holographic wall tomorrow? 

“Assuming that the content had any depth information with it that the studio provides, then yes,” Karafin says. “If it exists as 2D you can also convert it to 3D and display it in real time.” 

Holographic TV 

The SolidLight hardware includes a codec to deliver ‘visually lossless encoding’. “We will see increased efficiencies over the next few years that will further increase that performance,” he says. “A commercial broadcast format is where we are moving towards in the future.” 

Believe it or not the building blocks for holographic TV are emerging. Among them is a standard format for creating, processing and transmitting content which is being supported by IDEA (Immersive Digital Experiences Alliance), a group including Light Field Labs, and other autostereo, VR and holographic developers such as Otoy and Looking Glass Factory, along with cable networks Charter and Cox and cable industry think tank CableLabs. 

Currently, there is no standard nor formal specification that addresses the wide variety of display formats available, from XR headsets to advanced light field panels. This is the gap that IDEA intends to fill with its Immersive Technologies Media Format (ITMF) – a suite of royalty-free specifications that establish a baseline for an interchange of immersive media.  

“You need a format standard that allows you to stream into the home. We’ve been working on this with a number of big telcos. We’re planning for the future. We’re getting very close with 5G rollout and as more streaming bandwidth is available. Those are the things that need to come online to support live streaming holography to every device.” 

• Read more Building a greater volume of virtual production

Light Field Lab is also touting “aspirational” applications. These include holographic telepresence, a type of communication also being worked on by Microsoft and Google among others. 

This would require the system to be bidirectional and twice its current astonishing pixel density “because we’re not only projecting the holographic image in real time but receiving it also”, Karafin says. 

In theory, this is would be like looking “as if through a window 3000 miles away”. 

Another idea seems like closer to the science fiction of Black Mirror. Imagine if it were possible to playback our home movies as holograms? Since it’s perfectly possible to record video in 3D (or 180/VR) today this isn’t as far-fetched as it sounds. 

“There is no better way to see and capture our memories than to have that experience recreated directly in your living room,” Karafin says. 

Light Field Labs hopes to produce the panels in greater volumes, in turn reducing the price and making it more affordable to the wider market. Ultimately, the company envisages whole rooms – potentially the cinema of the future – as the holodeck: “all our panels, all around you, immersed in it”. 

“We see the metaverse as the encapsulation of all things that are virtual,” says Jeff Barnes, EVP Creative Development at Light Field Lab. “There is nothing more directly integrated into the virtualisation of the world than a true holographic display. VR and AR is a stepping stone that gets you into stereoscopic so you can move around and see portions of a virtual environment, but if you can replace physical things with nothing but photons of light that is our belief for what the metaverse truly has in store for the future of society.” 

Holographic filmmaking 

Could holographic video walls upgrade the LED displays currently used on virtual production soundstages? There’s no reason why not. In fact, their use would allow a DP to shoot without concern for moiré or chromatic aberrations, and without focusing on an LED screen.  

“If you had a holographic wall behind your actors – or you had holographic actors on set – you can photograph those objects or that background identically to having the real thing there,” Karafin says. “In other words, if you racked focus it would give you true depth of field. Your lighting of the scene would be based on that hologram projecting into space.” 

Holographic characters could be initially used on set to block scenes in pre-production. Another huge advantage, says Karafin, is enabling multi-camera capture.  

“You can’t do this today on volume stages where the background display is directly linked into the viewpoint of one camera aperture. But with holographic you can shoot as many cameras you like as long as it’s within the holographic field.”