Metaverse interoperability challenges and impact
An open metaverse that mimics how we operate in the real world depends on interoperability. What is interoperability, why is it important and what will it take to make it happen?
The metaverse implies a shared environment that spans a multitude of 3D virtual worlds. Participants in the metaverse will be able to move freely through these virtual worlds, taking their identities, entitlements and goods with them. At least that's one vision of how a universe of 3D virtual worlds will work.
In parallel, product lifecycle management, supply chain management vendors and others are laying the foundation for industrial metaverse interoperability. This lets digital representations of products, equipment, factories, warehouses and facilities be shared more efficiently to streamline industrial automation initiatives.
These industrial metaverse applications combine real-time, full physical fidelity digital twins with sophisticated visualization of manufacturing plants and processes to enable design simulations, process monitoring and optimizations, and forward-looking prediction and planning to increase efficiency and profitability, said Neil Trevett, vice president of developer ecosystems at Nvidia, and president at Khronos Group and Metaverse Standards Forum.
For example, Nvidia worked with BMW to build digital twins of its factories to optimize production lines ahead of new vehicle launches; Siemens Gamesa to simulate wind farms to maximize energy yield; and Ericsson to create digital twins of cities to place 5G towers and virtually fix problems.
Realizing this vision of an easily traversed, open metaverse, however, will require interoperability across multiple capabilities.
"Without standards and interoperability, there will be a flood of incompatible or isolated solutions slowing down the progression of data exchange between experiences," said Royal O'Brien, executive director at the Open Metaverse Foundation and Open 3D Engine (O3DE), which are both involved in coordinating work on open source standards for the metaverse and 3D.
Building and agreeing upon standards is typically an arduous process. "For a standard to be successful, there has to be a level of trust where one party cannot influence the final design to their advantage," O'Brien said.
Whether that can happen in the metaverse is an open question. Many of the current leading companies in the fledging metaverse have made their mark -- and make their money -- from having proprietary systems.
But while a walled garden approach might serve companies well in the short run, in the view of O'Brien and others, it will ultimately prevent the metaverse from blossoming, as explained below.
So, why is interoperability so important and what needs to be done to achieve it? That depends on how interoperability in the metaverse is defined.
What is metaverse interoperability?
The metaverse often gets conflated with the virtual reality (VR) and augmented reality (AR) technology used to experience a shared 3D world. Interoperability in the metaverse, however, is more than just the ability to share 3D models and visual avatars, said David Smith, founder and CTO at Croquet, a browser-based OS for the metaverse.
"It is essential that metaverse worlds are fully collaborative and easily connected via live portals, and that smart objects and components can be moved and reused across worlds," Smith said.
In an interoperable metaverse, a person's identity and ability to engage in commerce are as seamless as in the real world. Consumers can bring their wallets and smart objects to virtual worlds, just as they bring their credit cards and backpacks to stores today.
Attributes of the metaverse
The term metaverse, coined by sci-fi writer Neal Stephenson in his 1992 novel Snow Crash, generally refers to a shared persistent 3D digital world where people work and play.
Matthew Ball, author of what is widely considered one of the best primers on the metaverse, defines this world as having the following features:
- Massively scalable, interoperable, real-time 3D.
- A synchronous and persistent experience.
- Continuity of data, identity, history, entitlements, objects and payments across worlds.
That said, interoperability in an open metaverse will be a little more nuanced than in the real world and technically more challenging, as all the systems and standards to make it happen aren't yet in place.
Components of interoperability in the metaverse
The following five components of interoperability need to be addressed to create an open and easily traversed metaverse.
Identity
Identity forms the root chain of trust for connecting individuals to actions and assets. Today, consumer identity is often built on a chain of trust that is linked to email, social media platforms, Google or Apple. Business use cases in the metaverse will likely build on an enterprise's existing identity and access management infrastructure.
Standardization of identity and authentication, transactions and a standard means of data interoperability across multiple immersive platforms are essential aspects of metaverse interoperability in the short run, O'Brien said.
"Until I know who you are and how I can transact with you, as well as the means to hand you off to another server experience, you will break immersion," he said, referring to being jolted out of the illusion of being in another world or in a business context from the flow of work.
Smith added that identities are more than just the avatars representing a user. Every user will have multiple avatar representations and sometimes multiple ones in the same world. Identities help connect a wealth of information to an avatar, including not only the avatar's wallet but its preferences, history, reputation and abilities.
3D rendering
There are many levels to standardizing 3D representations of virtual worlds. 3D modeling and rendering engines need to interoperate. AR, VR and extended reality (XR) headsets need to work across platforms, and users need to share 3D objects across platforms and render them at scale.
Progress is being made. "Metaverses and digital worlds have been a forcing factor in our industry to standardize on high-fidelity 3D file formats that allow us to move data in and out of these worlds," said Jonathan Girroir, director of marketing at SDK provider Tech Soft 3D.
Promising approaches include the following:
- Graphics library transmission format (glTF) for sharing 3D objects.
- Universal Scene Description (USD) open file format for sharing 3D worlds.
- 3D Tiles to efficiently stream geospatial content at scale.
- O3DE for 3D object behavior interoperability.
- OpenXR for sharing across 3D VR and extended reality devices.
Behaviors and properties
There needs to be a standard way to characterize the behavior and properties of objects in the metaverse that translates across platforms. "These are the things that make the objects in the metaverse come alive and be responsive to the users," Smith said. To do that, behaviors and properties must be persistent, copyable and able to move between worlds.
In the industrial metaverse, this could include physical properties and the costs of components or materials in an object to automatically compute the bill of materials. Nvidia is championing its PhysX engine as one approach to describe physical properties in the industrial metaverse.
Data sharing
An open metaverse also requires a decentralized approach to sharing information across worlds. Decentralized ledgers such as blockchain have been championed as one approach. Early blockchain use cases have helped spark a market for nonfungible tokens (NFTs), which let people buy and sell assets like art and land on the web. Down the road, more sophisticated NFTs could enable complex revenue-sharing agreements. For example, multiple artists could each automatically receive a royalty when their songs are remixed into a new compilation that becomes popular.
In contrast, Tim Berners Lee, who pioneered the web, is advocating for a new web data infrastructure called Solid that doesn't use blockchain. It lets individuals control how their data can be reused after the fact. This could prove crucial for enterprise metaverse use cases that create value for consumers, patients and citizens while respecting the owner's changing wishes.
Generative AI
GenAI is accelerating the metaverse by driving API and model development and other metaverse content, according to Dan Isaacs, general manager and CTO at the Digital Twin Consortium (DTC). The original transformers that led to modern GenAI apps were built as a translator between French and English. Enterprises are also starting to use new variants to help streamline the process of translating data suitable for different representations and the enterprise apps that might connect to them. However, these tools need to be approached cautiously since they raise new concerns around security, trust, governance, data provenance and access.
Why is interoperability in the metaverse important?
The initial focus of an interoperable metaverse will be the consumer market, according to those interviewed, as it will be important for consumers to know they can retain the value of their virtual assets even if they change platforms. Ideally, the sneakers your avatar bought on one virtual platform should still be wearable on another.
Interoperability is necessary for consumers to have confidence their purchases work across platforms, said Yugal Joshi, an analyst at Everest Group. As others have argued, he said, it will ultimately also be needed to generate sales and establish a metaverse economy.
It's also important to consider industrial metaverse interoperability to improve the sharing and integration across digital twins of supply chains, products and the factories that build them with trusted business partners to improve collaborative planning and coordination or speed virtual training for new robots.
For more on the metaverse, read the following articles:
Metaverse vs. multiverse vs. omniverse: Key differences
Challenges with interoperability in the metaverse
Today, most immersive virtual worlds or digital twins live in discrete environments. A lot of development needs to be done to open those environments up and reduce the friction when moving applications across platforms, Girroir said. His team works on translating 3D data across various architecture, engineering, design and entertainment applications.
3D data from tools like AutoCAD and Blender and other 3D software form the foundation of the metaverse and will help businesses tie the digital with the real world, Girroir said. Enterprises will need to think about fusing lidar data captured in point clouds, vector data describing buildings and physics data describing how objects behave in the real world into a single, comprehensive view of a given asset or process.
There's also a growing interest in moving more of this kind of enterprise 3D data into the most popular gaming 3D engines from Unity, Epic and Nvidia. All these platforms come with faster, more realistic rendering capabilities than those available in traditional engineering tools.
Esri, the geographic information system giant, has developed SDKs for popular gaming engines from Unity and Epic to help enterprises render GIS data in 3D gaming engines. Meanwhile, Siemens and Nvidia have partnered to build the so-called industrial metaverse that aims to develop autonomous factories, improve product design and enable the next evolution of industrial automation.
While progress is being made, big challenges remain in the quest for metaverse interoperability, such as the following:
Sharing behavior
Sharing smart components, including avatar capabilities, is essential to the growth and expansion of the metaverse. In a gaming context, this might mean sharing information about the properties of things like swords or helmets across worlds. In an enterprise context, this includes describing how a machine works and connects to other things.
'Too many forums, but too little alignment'
Joshi said there has been a rise in interoperability groups for the metaverse. "This is a key challenge," he said. "Too many forums, but too little alignment." Every large player wants to shape the narrative in its own way without necessarily thinking about users, creators and communities at large.
For example, some people believe the metaverse is a collection of virtual worlds, so users should be able to traverse and converse across them. But even today, you can't use Microsoft Teams to speak to someone on Webex, or transfer money across different wallets or personas across social networks. "What the interoperability is trying to achieve is not clear and different participants are approaching this problem very differently," Joshi said.
Vendor-owned metaverses
Joshi also believes that most of the development is happening around vendor-owned metaverses. Interoperability across them is narrowly defined making it hard to know if assets in one metaverse are usable in other. This is a very vendor-centric view of the metaverse, which is unlike earlier aspirations where an interconnected metaverse would become like the internet, which no one owns. "The metaverse is becoming a wrestle between different vendors each wanting to own and define [it] in its own ways," he said.
Technical differences
Technical challenges, such as differences in 3D rendering engines, also hinder metaverse interoperability, Joshi said. He believes these differences can be overcome if there's a commitment to do so from vendors building the platforms.
Neglecting governance, risk and compliance
Joshi is also concerned that interoperability discussions are neglecting important enterprise considerations around security, governance, risk management and compliance.
"There is also a disturbing development where interoperability is considered to be the most important value element to be implemented," he said. This might lead metaverse builders to become shortsighted regarding other aspects of the metaverse such as security and compliance. Balancing these is a very complex process.
Persistent time
Ensuring persistent time across representations can also be a challenge, Isaacs said. This is crucial for digital twins that might operate at different time scales and level of details. For example, an electric meter digital twin might operate at minute time scales while a motor representation might have a scale of milliseconds. As data flows across different representations, it's important that time remains constant and translations don't cause it to speed up or slow down.
Coherent physics
Another interoperability issue, particularly for digital twins, lies in enforcing consistent physics across multiple digital twins. For example, a large vehicle, such as a trash truck, needs to account for the weight of its trash over a run and how that might affect the truck's performance and vehicle life across different roads.
Fall of digital collectibles
"The biggest setback to metaverse interoperability the last few years has been the fall of NFTs and digital collectibles," said Dr. Chris Mattmann, chief technology and innovation officer at the NASA Jet Propulsion Laboratory. This would have provided a unifying payment, collectibles and customization framework across multiple independent metaverses.
Interoperability in the metaverse: Around the corner or a pipe dream?
Mattmann said some of the biggest progress in terms of metaverse interoperability has been cloud providers such as Amazon Web Services and Meta going all in on metaverse frameworks like O3DE to render and store information about objects in the metaverse. Platforms built on O3DE are a good starting point for rendering and storing information about metaverse objects on a scale of millions of concurrent users. "Those platforms and companies will drive back-end standardization even though the front end of the metaverses will be different," he said.
Trevett said many industries are maturing in their approach to the metaverse, avoiding over-association with specific technologies such as virtual reality or blockchain. In addition, they're looking for ways to take advantage of advances in internet connectivity, spatial computing and AI to understand spatial environments, support intuitive user interfaces and streamline 3D content creation bottlenecks.
"Enabling multiple technologies and platforms to interoperate in the metaverse is going to take a constellation of shared open standards from multiple organizations," Trevett said.
He pointed to progress such as the following:
- The Metaverse Standards Forum is hosting wider industry participation and feedback on 3D asset interoperability and many other metaverse-relevant standardization topics.
- The Khronos Group, a nonprofit standard consortium, has been making progress on many metaverse-related standards such as OpenXR and the glTF 3D asset format.
- The Alliance for OpenUSD (AOUSD) and Khronos Group have signed a liaison agreement and are cooperating to ensure both standards achieve faster adoption using the strengths of each other's ecosystems, aligning on common data models and avoiding duplication and divergence that would cause industry confusion.
AOUSD's next milestone is to make OpenUSD perform well for real-time, large-scale virtual worlds and industrial digital twins. This includes building support for international character sets, geospatial coordinates and real-time streaming of IoT data. "Moving the state of the art forward requires both direct partnerships and collaborations among companies in this space to spur innovation, as well as careful curation of mature technologies into open standards that ensure long-term interoperability," Trevett said.
Many DTC members are making progress toward interoperability of industrial metaverse applications that span representations from different vendors, Isaacs said. He cited examples of companies such as Bentley Systems, Siemens, Navvis and Aveva working to improve interoperability with partners across their various platforms.
A more interoperable metaverse needs to contend with platform vendor efforts to lock in customers. Currently, vendors and application developers use different approaches for each interoperability domain. In a genuinely interoperable metaverse, every participant would use the same standards and formats in processes.
In the meantime, many metaverse use cases will work just fine with limited interoperability across a few of these domains. For example, someone could bring a pair of sneakers between worlds with specific and narrow agreements about identity and basic 3D rendering.
More sophisticated workflows, particularly in enterprise use cases, will require greater agreement on more aspects. For example, an industrial digital twin of a factory might have to bring together physical plant data using sensors from different vendors, 3D rendering to share data across various applications and tools, and identity management to enforce security across individuals and machines.
An open metaverse, if it happens, will take time. The industry took decades to develop standards for basic web interactivity and enable basic web applications and then mobile applications. Specifications for 3D interactivity and decentralized data sharing are still in the formative stages as enterprises discover the most promising use cases and profitable business models.
It might take another decade for interoperability to converge to where mobile is today. And in some cases, there might never be interoperability. For example, many mobile experiences are quite similar across Apple and Google devices. On the other hand, Apple has been slow to extend the Apple Messaging experience to other mobile devices. The metaverse will likely have the same nuances.
Apple has actively discouraged developers from using the terms metaverse, VR and XR to describe its new Vision Pro 3D camera, instead referring to it using the term spatial computing. Mattmann predicted Apple "won't drive interoperability there, or even on the back end, since Apple is an extremely risk averse and proprietary friendly rather than open source or open collaboration company."
George Lawton is a journalist based in London. Over the last 30 years he has written over 3,000 stories for publications about computers, communications, knowledge management, business, health and other areas that interest him.
