Activity Recognition using Visual Tracking and RFID (Krahnstoever 2005)

Summary:
The paper combines vision based tracking with RFID sensors to gain more information about tasks being performed. A minimum of two color cameras track the position of the head and hands of a subject by using skin color matching techniques. The subject's state is modeled by three ellipsoids whose centers are represented in spherical coordinates. The volume in which tracking takes place is divided into three parts, one for the head and one for each hand, to reduce the number of particles needed for tracking. The presence and orientation of key objects are tracked using RFID tags. To fully determine an object's orientation, three antennas must be attached to an RFID tag.
The information from visual tracking and RFID tags works together to identify activities of the subject. For instance, the presence of a user's hand in the emitter field of an object and the movement of an RFID-tagged object can imply that the user has picked up and is holding that object.

Discussion:
RFID-tagged objects are much easier to recognize than they would be using visual appearance alone. The system is able to track objects that would be occluded using visual information alone. One of the interesting applications suggested involved multiple RFID tags per item. If one were placed on the lid and one on the container, the action of opening the container could be detected.
When computing the likelihood of a pixel belonging either to a body part in the foreground of an image or to the background, each pixel in the projection of the union of the ellipsoids tracking the head and hands must be evaluated. Instead of performing collision detection on ellipsoidal volumes, rectangular bounding box volumes are used since their collisions are simpler to compute. This technique of reducing the computational complexity by simplifying collision detection boundaries is frequently used in the field of physically based modeling, and quite appropriate to use in this case.