Computer vision issues during eye-in-hand robotic tasks

Our previous work has introduced the framework of Controlled Active Vision that provides a bridge between computer vision and control theory. This paper presents some of the computer vision techniques that were employed in order to detect targets, automatically select features, measure features' displacements, and evaluate measurements during the various applications of the Controlled Active Vision framework. We experimented with a lot of different techniques, but the most robust proved to be the Sum-of-Squared Differences (SSD) optical flow technique. Search optimizations and dynamic pyramiding are proposed in order to provide real-time performance. In addition, several techniques for the evaluation of the measurements are presented. One important characteristic of these techniques is that they can also be used for the selection of features for tracking in conjunction with several numerical criteria that guarantee the robustness of the servoing. These techniques are important aspects of our work since they can be implemented either on-line or off-line. The paper concludes with results from the application of these techniques to real images.