Into the Metaverse

A Unified Theory of Computing

The term “Metaverse” generally invokes nebulous notions of a virtual world which extends or even replaces the physical one in which we live. Depending on our views, we may see the metaverse as the marker for a dystopian and cold future, or be wildly excited by the economic possibilities enabled by rich, real-time, immersive experiences which transcend physical boundaries. Regardless, it is difficult to deny the metaverse is already under construction and—like previously nebulous concepts in tech (e.g., the internet, the cloud, artificial intelligence, etc.)—it is likely to be driven by such formidable economies of scale that it will be impossible to stop. 

So what is the metaverse, and which (atypical) problems are the most interesting and crucial to solve?

📚 What is the Metaverse?

Although there are plenty of articles online describing the metaverse, these two provide compelling narratives: The Metaverse: What it is, Where to Find it, and The Metaverse Value-Chain. They show the metaverse will likely be a persistent, real-time, and highly flexible set of physical and digital experiences one can participate in alone or with a group, all within a fully functional economy. Conversely, the metaverse will not just be a virtual world, a game, or digital commerce platform. It will be all those things, and much more.

Even when armed with the above definitions—real-time and persistent digital experiences, fully functional economies, allowing single or group participation—it is still very difficult formulate a concrete idea of what the metaverse is/will be. That’s because these ideas are still vague, and we have yet to draw a proverbial bounding box around them to understand what they really mean. Instead of focussing on these, we analyze the metaverse from first principles (simple, well-known concepts in computing). We look towards understanding computers more generally, and then work backwards. 

♾️ Encoders and Transducers

A computer can be thought of as a general purpose encoder¹ of our physical world to bit-space, and a transducer of physical labour to electrical energy. In essence, when we use a computer to perform any task, the first step is to ensure the relevant information from our physical lives is encoded to a digital domain (bit-space). Then, we use electrical energy to transform, transport, and consume this data in ways that are economically beneficial. As an example, consider the transformation that computing led within the film industry value chain. 

Initially, video was recorded on film, edited by hand, and then physically transported to viewing areas for public consumption. Computers essentially allowed us to encode video to bit space, wherein we massively improved the efficiency, scale, and types of edits performed. We were then able to distribute and consume the content from wherever, whenever (so long as the relevant infrastructure was in place). The required energy spent transforming and transporting physical materials was all transduced into electrical energy, which is much more efficient. 

The digital world, inherently, has a much higher spatial density² than the physical world, and requires only one type of highly-efficient energy (electrical) to operate³. This fundamental view of computers as highly scalable and efficient encoders and transducers offers another view into why technology companies are able to become such strong aggregators and are so economically successful. We can use this to view to understand the metaverse.

🪨 The Next Milestones in Compute

Since their advent, computers have been used to transform our physical world into a much more efficient digital one. Changes have been continuous, while certain core pieces of infrastructure (personal computers, applications, smart phones, the internet) have led to multi-decade transformations that continue to unfold in unpredictable ways. An argument can now be made that the metaverse, as a term, simply represents the next batch of milestones in this general digitization. The next set of milestones will be centered around using our physical world as a base layer atop which to build immersive digital constructs. Some recent advances in technology hint strongly in this direction. These advances include, and are not limited to: 

• Useful health and fitness trackers (article)

• Smart home devices (article) and smart speakers/displays

• Rapid growth of AR/VR (article) and the addition of 3D mapping hardware to phones (article)

• Improvements in commercial telepresence tools (article) 

• Advancements in distributed consensus through blockchain

The technologies above are not concepts or thought experiments, but are actively being built and in various stages of maturity/acceptance by the market. It’s clear that although there is a long way to go, the metaverse is already well under construction.

💾 More Natural Computers

This metaverse revolves around building computers with which we can interface more naturally using voice, touch, gesture, and even body-language, which are all interconnected, and move towards unified interfaces. These computers are and will increasingly be unconstrained by their hardware, being able leap from their physical form to overlay the world in which we live. Another somewhat under-appreciated aspect of this metaverse is the trend towards measuring our activity and recording the enormous amounts of data we generate. Massive increases in data capture naturally lead to the democratization of complex analysis and modeling techniques. These techniques are aimed at better understanding human activity to help people broaden their productive and leisurely pursuits. For example, modern fitness trackers can now measure, model, and inform people on everything from gait to sleep quality to athletic performance—insights which only a few decades ago were reserved for the most elite athletes.

🙋 Open Questions and Challenges

Continuing to build on these ideas, we reach a set of open-and-interesting questions that hopefully inspire thought on how to build a metaverse which is safe, functional, and works for everyone. These problems are in no way all-encompassing, but represent what I find to be the most interesting challenges which require the most thought.

Which bridges should we build?

Bridges are connectors between the physical and digital worlds (we discussed various examples in this post). As we develop more of the metaverse, it’s worth considering which bridges should be built next, how should we interact with them, and how should their interfaces be unified such that they can be crossed with a minimal learning curve?

Can we use the metaverse to minimize our environmental footprint?

The metaverse can be viewed as a transformer converting physical experiences to digital ones, thus offering a unique opportunity to minimize our energy and material footprint on the planet. If we are able to transform more economic and leisure activities to bit-space, we can power them all through electrical energy, and drastically reduce their material costs and waste outputs. As the compute backbone of the metaverse becomes unified and efficiently managed by large-scale cloud providers, these providers’ efforts to reduce resource consumption go a long way in making it more sustainable. 

Somewhat controversially, a related open question comes to mind around central nervous system stimulation, and the role this will play in powering digital experiences which are even more immersive—stimulating more than just visual and auditory senses. Which physical and psychological wellness best practices should be developed guide the development of such experiences?

How do we responsibly model and analyze the data generated in the metaverse?

Finally, as the amount of data we capture and generate continues to grow rapidly, advancements in machine learning will accelerate to better model and process this data. As an example, large scale image datasets have motivated the development of highly versatile computer vision systems. Similarly, large scale auditory and textual data have enabled the development of transcription, translation, and speech synthesis systems. However, these models narrowly capture human capabilities, and require specific and manual task specifications to function. 

How will we represent multi-modal datasets to enable the modeling of more complex behaviours and not specific tasks? How will the narrow capabilities of machine learning today be woven together to not only model the world, but explore it without manual objective specifications? How will we benchmark these capabilities against human ones in understandable ways? Lastly, how will we make these programs available to everyone?

🔚 Final Thoughts

In conclusion, as we pull the curtain on the metaverse, we see that it is simply the next step in computing. We see that many of the current popular trends in technology (AI, Blockchain, AR/VR, etc.) are all a part of the next technological transformation and the development of the metaverse. We also note that the unified view presented here provides a mental model through which to understand and evaluate new technologies. Finally, answers to the list of open questions regarding the metaverse likely present opportunities to build the next set of powerful businesses in tech.

1. Encoders in this context are devices that convert one form of information into a coded form, usually to achieve some benefit in storing, transforming, or transferring that information. 

2. The information density of digital computers is orders of magnitude higher than that of physical, human readable stores of information. Paperback, a curious technology to backup digital data on paper, claims to store up to 3MB of data on a single A4 sheet. This is far from human readable, but if we run with this concept, a stack of 100 A4 sheets would store 300MB of data. Solid state drives, and other modern storage mechanisms, offer storage densities of multiple terabytes per square inch. This back-of-the-envelope computation shows a multi-order-of-magnitude difference.

3. Note that we focus on operation. Of course, there are still other forms of energy used during the production, distribution, and maintenance of the necessary hardware and tools.