GWindows: Light-weight Stereo Vision for Interaction

Overview

GWindows explored computer interaction at a distance, where a mouse and keyboard are awkward—for example, when controlling a media display. Earlier perceptual interfaces were often computationally demanding, sensitive to conditions outside the laboratory, or dependent on careful calibration.

The prototype used two inexpensive video cameras to estimate depth. That depth signal made foreground detection and hand tracking more robust, while speech recognition supplied an additional input channel. Together, gesture and speech supported window movement, resizing, selection, and freehand drawing across a multi-monitor desktop.

The work demonstrated that lightweight stereo could support these interactions in real time and included a preliminary user study. The evidence remained limited to a prototype and an early study, however; it did not establish how reliably the interaction would work over long periods, for a broad range of users, or in varied rooms and lighting.

GWindows in use: stereo vision interface, multi-monitor setup, moving windows by hand, and ink-drawing.

Publications

Conference paper
PDF DOI
Cite
Formatted citation

Andrew Wilson, Nuria Oliver (2003). GWindows: Robust Stereo Vision for Gesture-Based Control of Windows. Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI 2003), 211-218. https://doi.org/10.1145/958432.958473

BibTeX
@inproceedings{wilson2003gwindows,
  author = {Andrew Wilson and Nuria Oliver},
  title = {{GWindows}: Robust Stereo Vision for Gesture-Based Control of Windows},
  booktitle = {Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI 2003)},
  pages = {211--218},
  year = 2003,
  doi = {10.1145/958432.958473},
  cites = 34,
  citesdate = {2026-03-23}
}

GWindows: Robust Stereo Vision for Gesture-Based Control of Windows

Andrew Wilson, Nuria Oliver
Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI 2003) · 2003
Conference paper
External link
Cite
Formatted citation

Andrew Wilson, Nuria Oliver (2005). Multimodal Sensing for Explicit and Implicit Interaction. 11th International Conference on Human-Computer Interaction (HCI International 2005). https://www.microsoft.com/en-us/research/publication/multimodal-sensing-explicit-implicit-interaction/

BibTeX
@inproceedings{wilson2005multimodal,
  author = {Andrew Wilson and Nuria Oliver},
  title = {Multimodal Sensing for Explicit and Implicit Interaction},
  booktitle = {11th International Conference on Human-Computer Interaction (HCI International 2005)},
  address = {Las Vegas, Nevada, USA},
  year = 2005,
  url = {https://www.microsoft.com/en-us/research/publication/multimodal-sensing-explicit-implicit-interaction/},
  cites = 31
}

Multimodal Sensing for Explicit and Implicit Interaction

Andrew Wilson, Nuria Oliver
11th International Conference on Human-Computer Interaction (HCI International 2005) · 2005

Videos

Press