Graphical Models for Driver Behavior Recognition and Prediction in a SmartCar

Nuria Oliver and Alex Pentland — MIT Media Lab / Microsoft Research

Abstract

We developed a SmartCar testbed platform: a real-time data acquisition and playback system combined with a machine learning framework using dynamical graphical models for recognizing driver maneuvers at a tactical level, with particular focus on how contextual information affects driver performance. The SmartCar's perceptual input is multimodal: four video signals capture surrounding traffic, the driver's head position, and the driver's viewpoint; a real-time data acquisition system records brake, gear, steering wheel angle, speed, and throttle signals.

Over two months, more than 70 drivers operated the SmartCar for 1.25 hours each in the greater Boston area. HMMs and CHMMs were trained on this experimental data to create models of seven driver maneuvers: passing, changing lanes left and right, turning left and right, starting, and stopping. These models are essential for building more realistic car simulators, improving human-machine interfaces in driver assistance systems, and preventing dangerous driving situations.

Publications

A Graphical Model for Driver Behavior Recognition in a SmartCar Nuria Oliver and Alex Pentland. Intelligent Vehicles 2000, Detroit, Michigan, October 2000.

Driver Behavior Recognition and Prediction in a SmartCar Nuria Oliver and Alex Pentland. AeroSense 2000 / Enhanced and Synthetic Vision, Orlando, April 2000.

Videos