Nvidia: The metaverse is coming

We are entering the age of AI, one which promises super-scale virtual worlds, autonomous vehicles and a swift end to global pandemics. It’s one in which artificial intelligence actually writes software and Nvidia claims to have cracked the code. 

The tech company is best known for making specialist graphics processors for gamers. Recently it became the most valuable American chip company, darling of investors and a major force in robotics, scientific computing, data centres and the 5G edge. Its AI smarts will soon potentially be inside billions of smartphones, servers, PCs and consumer electronics. 

That follows the company’s U$40 billion swoop for British chip manufacturer Arm Holdings from Softbank. The proposed takeover would be the silicon industry’s biggest deal ever. If ratified (and there are anti-competition hurdles to jump) the deal would pose a formidable threat to Intel, according to the WSJ. 

Apple, Qualcomm, Microsoft and Samsung are among companies licensing designs from Arm.  

“A few weeks ago we announced our intention to acquire Arm, the most popular CPU in the world,” said chief executive officer Jensen Huang, during a keynote to the virtual GPU Technology Conference earlier this week.  

“Together we can offer Nvidia accelerated AI computing technologies to the ARM ecosystem reaching computers everywhere.” 

Huang is not averse to hyperbole. The Taiwanese-American founded Nvidia in 1993 with just $40,000. Its designs for accelerating the graphics quality for video games brought instant success. In 2006, its invention of a programming model that enabled multiple complex mathematical calculations to run in parallel made its graphics processing units (GPUs) a staple in cloud computing, AI, and data. It has shipped over 1 billion of its chief GPU called CUDA.  

Its software development kits has been downloaded 20 million times. There are 6,500 start-ups building on Nvidia and more than 2 million Nvidia developers.  The number of its GPUs used by major cloud providers like AWS and Azure now exceeds that of all cloud CPUs.  

“Within three years, Nvidia GPUs will represent 90% of the total cloud inference compute,” he said. “We are passed the tipping point.” In July, the company overtook Intel on the NASDAQ to reach $251 billion. 

So, let’s take the hyperbole seriously. “If the last twenty years was amazing, the next twenty will seem nothing short of science fiction,” he prophesied. “The metaverse is coming.” 

First coined by author Neal Stephenson in 1992 novel Snow Crash, and elaborated by sci-fi like Ready, Player One, the metaverse is conceived as the next generation internet.  

“It is one where humans as avatars and software agents interact in a 3D space,” said Huang. “A VR successor to the internet.” 

Games building worlds like Minecraft and Fortnite presage the beginning of the metaverse. “Though they seem like games today, inhabitants of these early metaverses are building cities, gathering for concerts and events and connecting with friends,” said Huang. “Future worlds will be photorealistic, obey the laws of physics (or not) and be inhabited by human avatars and AI beings.” 

Not just a place to game, the metaverse “is where we will create the future… before downloading the blueprints to be fab’ed (fabricated with 3D printing) in the physical world,” he said.  

Nvidia’s ambition is similar to Magic Leap, Google and Apple which have also articulated versions of a 3D internet, populated by avatars, governed by spatial computing. It wants a controlling share of this online future by launching Omniverse. 

• Read more: Leaping into a new reality 
  

Described as a remote platform for simulation and collaboration, the Omniverse “fuses the physical and virtual worlds to simulate reality in real time with photorealistic detail,” Huang said. “The Omniverse is a world where AI agents are created. It’s a sim for robots. A place where robots can learn how to be robots.” 

Cloud native and photoreal with path tracing and material simulation, the Omniverse allows designers and artists and even AI’s to connect in a common world. “This is the beginning of the Star Trek Holodeck, realized at last.” 

Now in open beta, the tech has been evaluated by users including Industrial Light and Magic and is based on technology originated at Pixar called Universal Scene Description. This is a format for universal interchange between 3D applications from Epic (Unreal Engine), Adobe, and Autodesk (Maya).  

Virtual production technology company Lightcraft Technology has also evaluated the platform. Its chairman, Bill Warner (who is also the founder of Avid), says they decided to base their entire product line on the technology. 

“Omniverse represents the platform of the future for all aspects of virtual production,” Warner said. 

Francois Chardavoine, VP of technology at Lucasfilm and ILM testifies of omniverse: “The potential to improve the creative process through all stages of VFX and animation pipelines will be transformative.” 

However, even the omniverse and the metaverse pales beside the firepower of AI. According to Huang, “As mobile cloud matures, the age of AI is beginning. AI is the most powerful technology for of our time.” 

He argued, “Computing is the technology of automation. Software algorithms automate. Automation drives productivity and growth for industries. Automation at large scale leads to breakthroughs. [But] we are limited by our ability to write software. Finally, after decade of research and deep learning, the abundance of data and the powerful computation of GPUs came together in a big bang of modern AI. Now software can write software. AI is the automation of automation.” 

He acknowledged that the software written by an AI is very different to that written by a human (“it is vastly more parallel and a thousand to millions of times more compute intensive”) but claimed the ability to power the computers needed for it are in Nvidia’s wheelhouse. 

“AI requires a whole reinvention of computing, from algorithms to the whole ecosystem. The age of AI has begun and Nvidia is advancing it.” 

An example of its god-making power is medical research. He suggested that with AI powered by Nvidia GPUs scientists can use simulation to magnify the search for a cure to diseases.  

He announced Nvidia would build the fastest supercomputer in the UK. Cambridge One, in Cambridge, would have 400 petaflops of AI performance putting it in the top 30 supercomputers in the world. Research partners include Astra Zeneca and the NHS. 

It recently launched Ampere, a GPU which fuses programmable shading, ray tracing and AI capable of generating photoreal graphics at the highest frame rates, solving the traditional trade off which video game developers had to make. 

“It’s the faster upgrade in our history. Everything we ship is instantly sold out.” 

“If the last twenty years was amazing, the next twenty will seem nothing short of science fiction…the metaverse is coming.” Jensen Huang, Nvidia 

He also launched a cloud streaming video AI platform whose first application is video conferencing. This will take advantage of a Nvidia-developed ‘Conversational AI’ called Jarvis whose neural text-to speech is described as “human like.” 

“Using AI we can perceive the important features of a face, send only the changes of the features and reanimate your face at the receiver – to save bandwidth,” he said. “AI can reorientate your face to look as if you’re making eye contact with each person on the call individually. It can realistically animate an avatar based on just the words you are speaking.” 

He added, “We have an opportunity to revolutionise video conferencing and invent the virtual presence of tomorrow.”  

More than 30,000 developers have registered for the virtual GTC to check out more than one thousand talks on topics including autonomous machines, VR and virtualisation. There are also webinars related to latest AI developments in media and entertainment driven by Nvidia tech. Among them is cloud-based tool Kamua which automates resizing, cropping and repurposing videos from desktop to mobile apps like TikTok, Snapchat and Instagram. It claims to have users who have cut manual workflows from 16 hours to 15 minutes by embracing its automation software. 

IDenTV provides advanced video analytics based on AI capabilities powered by computer vision, automated speech recognition and textual semantic classifiers. 

Another demo from Taiwan start-up A.V. Mapping shows how AI enables composers to use a its video and music mapping platform “to shorten the music-for-video search by nearly two thousand times.”       

• Read more: The future of immersive content