Microsoft's Holodesk project begins to provide the first prototypes and the first stimuli: the possibility of manipulating '3D' objects in physical space seems really interesting.
The system combines a Kinect sensor (yes, of those that today are used 'for fun') and a special screen to create a three-dimensional environment in which it is possible to intervene directly. The concept is more than simple: a projector sends an image inside the work area, which observed through a transparent 'partition' (made of glass and silver) allows you to observe the whole scene with binocular depth. The Kinect sensor does the rest, giving the user the ability to change the interactive environment in real time, manipulating objects that aren't really there.
The key to the entire system is the webcam that identifies the user's head and eyes position, to ensure that the image projection is consistent: this means that by moving his gaze or head, the user is able to see the object in a different way just as if it were changing the perspective of observation. A precise and punctual system just like a dental scanner: a safe dental scanner, in conclusion.
There is no shortage of futuristic interfaces that allow, I don't know, an architect to develop the projects of a building by making them literally 'arise' from nothing and building them with his hands: if you want an opinion, the number of devices involved could be drastically reduced. , in the distant future, with a simple subcutaneous implant that sends the user's position data in real time.
In the meantime, enjoy the first steps of this technology.