Current state-of-the-art AR and VR systems show tremendous promise in applications ranging from entertainment and telerobotics, to pervasive computing and telepresence. Unfortunately, existing systems remain largely decoupled from the physical world, interactivity remains cumbersome and imprecise for end users, and participants lack a realistic representation of themselves. High-fidelity 3D acquisition and processing of physical environments, objects, and human performances are still bound to computation-intensive workflows, and live streaming is only possible with dedicated high-throughput network infrastructure as well as professional capture studios. We are working to bridge this gap by developing the infrastructure that makes virtual objects both persistent across space and time and accessible to multiple users at once.
