Online, Interactive Learning of Gestures for Human-Robot Interfaces (Lee 1996)

Summary:
This paper's goal was online teaching of new gestures with few examples utilizing discrete HMMs to model each gesture. The system uses a CyberGlove to recognize letters from the sign language alphabet. Input data is sampled from 18 sensors at a rate of 10 Hz and then reduced to a one-dimensional sequence of symbols using vector quantization. The vectors are encoded as a symbol from a codebook, which is a set of vectors representative of the domain of vectors to be encoded. The encoded vectors are sent to a Bakis HMM where the gesture, whose probability is above a certain threshold and not too near other gestures, is chosen as the one recognized. Segmentation is handled simply, by requiring the operator to hold the hand still for a short time between gestures. Only 14 letters were used, and the letters that were chosen are relatively easy to distinguish from each other compared to all possible sets of 14 letters. After two training examples, two experiments yielded 1% and 2.4 % classification errors.

Discussion:
In the section describing the interactive training, the condition of the system being "certain" about a classification of a gesture is described as being handled by performing the associated action. The specifics of being "certain" are not given, but could have been reported as a threshold probability of observed data matching the sequence of observations describing a gesture.
At the end of the interactive training section, the scenario of specifying a new "halt" action is outlined. The details of how the user specifies the label and action associated with a gesture recognition are not given. This guidance required by the user seemed to raise the difficulty level for interacting with the system.
The choice of Bakis HMMs do work for the gesture set of basically static ASL letters, but would be poor for classifying repetitive motions such as waving. This system could not be generalized to recognize all types of gestures since the states of Bakis HMMs are restricted to moving forward.
Since the codebook is generated offline before recognition begins, is the correlation of the codebook vectors to observed vectors decreased when new gestures are being learned? Teaching new gestures will probably change the domain of vectors to be encoded. This should probably alter the codebook, but it does not.