Distributed Sensory and Motor Management

Distributed Perception

In collaboration with the UMass Vision Lab we have developed a variety of techniques for use in combining information derived from multiple, spatially distributed sensors. We have created several techniques that use redundant and spatially distributed arrays of sensors to track features through occlusion, to discover features that discriminate between multiple control contexts, and to respond at run-time to faults. Panoramic Image of LPR
To accomodate mobile sensors, we developed mechanisms for calibrating virtual sensors on the fly and planning sensor trajectories to improve the precision of localization processes.

Panoramic Image Processing

Some of our vision sensors are equiped with Panoramic Annular Lens (PAL) to produce images over a complete 360 degree panorama. These sensors can be posted in stationary positions or carried on our RWI and uBot platforms. While far field resolution is not great with optics like this, the field of view extends roughly from the horizon up to nearly the north pole of a viewable hemisphere. The image shows a panoramic image of the LPR at UMass. When it's mapped from the surface of the hemisphere onto a plane it looks like this.

Single Subject Pursuit with Redundant, Heterogeneous Sensors

Redundant arrays of heterogeneous sensor types allows viewpoints to be varied in order to accomodate motion trajectories that are occluded from any single stereo perspective. Moreover, other quality concerns like stereo conditioning and hardware/network faults can likewise be acommodated. The potential also exists for choosing sensor types that differentiate a subject from other ambiguous motion cues. For instance, a heading to a sound or thermal motion in addition to visual features may be a robust means of distinguishing humans and robots. Although panoramic cameras enable real-time motion tracking over a wide field of view, the low resolution of the image makes it very difficult to determine the identity of people or objects that we are tracking.

panoramic camera 1
panoramic camera 2

PTZ "close-up" front view

This is where another sensor modality, like a PTZ camera, can be used fruitfully. After triangulating using the heading reports derived from panoramic tracking service, the three dimensional location of the subject can be estimated. Having done so, this location can transformed into a gaze heading for a PTZ, which saccades to the target location and zooms to the estimated range to acquire a high resolution, narrow field-of-view detail of the subject. If the subject's trajectory is tracked over time, we can further guess whch direction the subject is heading and recommend the best PTZ to use to acquire a face shot.

Multi-Subject Tracking

To track multiple subjects continuously, it is necessary to both segment and differentiate the subjects. The first task we accomplish using a well known background subtraction algorithm that identifies moving subjects robustly and in real-time. The differentiation task is accomplished by extracting distinctive color histogram models of the subjects when they first appear as motion tracks that is used verify their identity in later frames.

An animated GIF illustrates a tracking result for two subjects that execute a challenging path that crosses.

Animated GIF showing the tracking of two people

(Click on the image below to play)

Animated GIF showing the tracking of two people
(Click on the image below to play)

More examples and details regarding methods for dynamically calibrating mobile panoramic sensor, error analysis and motion control for mobile sensors, and fault recovery using the Containment Unit (CU) abstraction mechanism can be found at Deepak Karrupiah's Distributed Sensor Networks page.