The Watching Window

Computers have looked much the same for nearly twenty years. Weare so used to the screen, keyboard and mouse that we forget that thisappearance is merely a fashion and an accident of history. We shouldbe able to communicate with a computer by speech and bygestures. Instead of using special tools like a keyboard, the computercan be programmed to determine our desires by watching an listening.

The Watching Window is an experiment in this idea of naturalcommunication. The user is « watched » by two tiny TVcameras and the computer must deduce from gestures what the user srequirements are.

In this simple demonstration, the computer is simulating a windowinto a 3D world. The TV cameras track your hands and eyes and thedisplay is changed accordingly. If you move your head, you see thesimulated world from a different angle. You can interact with thedisplay by pointing at simulated objects. The 3D effect is made evenmore convincing with the use of stereo glasses.

Hand and Eye Tracking

We are developing software to find and track human hands and eyes in a general way. In this demonstration, we separate the human figure, foreground, from whatever was in the camera view before, background. This separated image is a silhouette of the human figure. Our software attempts to identify the likely positions of head and hands from this silhouette.

Displaying a 3D World

As you move around a real object, you see it from different sides. You can look at objects in the watching window the same way. Lower your head and you get a view from underneath. To achieve this the computer must work out what the object would look like from your point of view. This is what we display on the screen.

Links

• Papers. A list of papers,theses and publications related to the Watching Window project.
• InnovateConference. Photos of the Watching Window display at the InnovateConference held in Christchurch in March, 2002.