UC San Diego

In this thesis, we investigate ways of enabling intelligent systems to recognize human desires and wants, and we devise systems that automatically recover human pose and gesture information. We place special emphasis on applications for improving the safety and comfort of vehicles. We present a novel method for learning and tracking the pose of an articulated body by observing only its volumetric reconstruction from images. The model is called the kinematically constrained Gaussian mixture model (kc-gmm). Pairs of components connected at a joint are encouraged to assume a particular spatial configuration, forming joints with 1, 2 or 3 degrees-of- freedom (DOF). Pose learning is based on the EM algorithm, and is the first to be evaluated using a common human image data-set with optical motion capture ground-truth. The algorithm achieved estimates with mean joint position error of 15.9cm, or 8% of the total length of the body. On synthesized hand data, the error was 0.5cm, or 1.5% of the total length. Next, we present results on the characterization and recognition of driver intent using driver gestural cues. The concepts apply towards the study of other driving maneuvers. The data-driven pattern classification approach makes use of vehicle dynamics information and driver head and hand pose information via an optical motion capture system. We present results comparing different combinations of input cues. We proposed a novel visualization of results to analyze the classifiers: ROC Area vs. Decision Time and Statistical Response Over Time plots. Driver-intent recognition algorithm above assumes the use of body part position information. We present an in-vehicle system for detecting and tracking the position of the left and right hands in long-wavelength infrared imagery. The results were effective in tracking hands over 90 minutes of driving. Combined with steering information, 5 hand activities over the steering wheel could also be determined. Finally, we present an in-vehicle system for determining which occupant is accessing the vehicle infotainment controls for modulating information flow from the vehicle's information display. The average correct classification rate of 97.8% was achieved over 60 minutes of 30fps video under a variety of moving vehicle operating conditions