Holoportation™
Since Holoportation is still very much in development, there’s not much that I can share directly, and while I can’t talk about product specific details, I’m happy to engage with folk about the general direction and needs that I have seen emerge working in this space for nearly 15 years. The quality of transmission of honest signals is key for meaningful human communication, and the authenticity of how we are represented is key to that quality of signal. Audio and 2D video generally work well. The evolution into spatialized experiences demands solutions that add to the quality of transmitting honest signals, it’s not enough to just replicate a 2D experience in a 3D space. Any barriers or friction costs to that communication must be handsomely rewarded in the experience. That’s a tall task.
Our group focused on the product side of Holoportation™, building a prototype ‘studio-in-a-box’ that we used not only for internal testing, but also for some limited real-world events, similar to Ms. White’s keynote.
The dream of real-time 3D telepresence for immersive mixed reality experiences is on the verge of coming true. For the last three years I’ve been part of a distributed team in Microsoft focused on the technology and experience requirements for Holoportation™. My work with the team has been focused on defining the balance of what the technology is capable of, and how to build a meaningful and delightful product that provides value for enterprise customers.
The dream looks something like this vision piece my group (Mixed Reality Capture Studios) helped produce the holographic content for. In this video former Microsoft executive Julia White shows herself delivering a high-quality holographic keynote in Japanese with the aid of AI enabled translation, trained to sound like her actual voice:
The image just above on the left was from the prototype broadcast rig being used at an event in Europe in 2022. AltspaceVR was used to support Holoportation prototyping and investigation, but now Holoportation is part of Microsoft Mesh, under development.
The initial prototype broadcast studios have been sent across the country and across the Atlantic to Europe. The learnings from the prototypes were used to dramatically strip down technical requirements, making a simpler, more robust solution that moves closer to fulfilling the expectations set in videos like the Julia White demo.
An example of a hybrid keynote event with a live audience, and a live 3D result being simultaneously broadcast into AltspaceVR. This set-up requires a minimal number of cameras and creates a nearly unobstructed view of the presenter for the audience.