Microsoft Research often comes up with interesting and sometimes mind-blowing projects. A recently published Microsoft Research project called SurroundWeb is an interesting one. SurroundWeb is a 3D web browser that will allow web pages the ability to display content across multiple surfaces in a room. For example, if you are watching a car race on your TV, the current driver standings can be shown on the side next to your TV, or an Wikipedia entry of a place which you are looking on the screen and more. Microsoft is trying to project this plan with no.of use cases. Read the abstract below.
We introduce SurroundWeb, the ﬁrst least-privilege platform for immersive room experiences. SurroundWeb is a “3D Browser” that gives web pages the ability to display across multiple surfaces in a room, adapt their appearance to objects present in that room, and interact using natural user input. SurroundWeb enables least privilege for these immersive web pages by introducing two new abstractions: ﬁrst, a Room Skeleton that enables least privilege for room rendering, unlike previous approaches that focus on inputs alone. Second, a Detection Sandbox that allows web pages to register content to show if an object is detected, but prevents the web server from knowing if the object is present.
SurroundWeb provides three privacy properties: detection privacy, rendering privacy, and interaction privacy while simultaneously enabling Web pages to use object recognition and room display capabilities. Surveys show the information revealed by SurroundWeb is acceptable. SurroundWeb is practical: After a one-time setup procedure that scans a room for projectable surfaces in about a minute, our prototype can render immersive multidisplay web rooms at greater than 30 frames per second with up to 25 screens and up to a 1440×720 display. We demonstrate a range of previously proposed and novel experiences can be implemented in a least-privilege way using SurroundWeb.
Find the full paper from the MSR link below. At last year’s CES, Microsoft demoed a similar concept for Xbox gaming console called Illumiroom. IllumiRoom uses a Kinect for Windows camera and a projector to blur the lines between on-screen content and the environment we live in allowing us to combine our virtual and physical worlds. This system can change the appearance of the room, induce apparent motion, extend the field of view, and enable entirely new game experiences.
This system uses the appearance and the geometry of the room (captured by Kinect) to adapt the projected visuals in real-time without any need to custom pre-process the graphics. What you see in the videos below has been captured live and is not the result of any special effects added in post production.