System for automatically detecting and tracking objects using a visually-guided manipulator

This paper presents a framework for the automatic visual detection of moving objects in robotic servoing tasks. The paper describes a 'figure/ground' scheme which is able to perform detection without making many assumptions that would limit the generality of the approach. We describe optimizations which allow for real-time execution of the frame-differencing that is the basis of our framework. Experimentation has demonstrated that the results of the visual detection can provide information helpful in focusing attention on a specific subset of objects. Furthermore, we show how the detection technique can be integrated with existing methods for visual tracking. We mention some details that have been addressed in order to apply these theories to an actual robotic system, including the use of an optimal controller. Our paper explains how this framework has been implemented in our experimental robotic system MRVT, and it describes several results obtained from this experimentation.