For the annual robot-contest at DTU (RoboCup) in 2004 there were constructed a mobile robot with an embedded stereo vision system used for navigation. More information about the RoboCup-contest can be found
The software taking care of the vision navigation is coded in GNU C++ with help from the OpenCV-library, and executed on an embedded PC running Linux. The software consists of a number of layers on top of each other. The lowest layer grabs the raw images from the two webcams (stereo). From the top layer navigation data such as direction and speed needed to follow a wall or find a gate can be read.
Generally the model consists of a CNavigator-object, consisting of two CEye-objects. Each representing one eye for the photogrametry measurements. The raw picture from the webcam is first sent through a specially designed imagefilter. Where it is transformed into a more simple picture without unnessesary details. In other words the picture is transformed into an array of hue-values (CCartoonImage).
By having the picture in this format, it is now easier to extract edges and objects in the image.
The simplification of the images is improved one final time by assembly of the edge-lines into 2D polygons. Each polygon consists of a number of edge-lines encircling the same object in the picture. In this way a polygon consists of a set of edgelines and a common color, being the most frequent of the object. The layer taking care of this is called the Polygon-layer. It also consists of a filter removing all polygons having colors without interest.
When both "eyes" (CEye-objects) have found a set of polygons for their image, the stereoscopic process can begin. This process is also called photogrametry and is executed in the Photogrametry-layer. Normally automated photogrametry system are very are hard to implement succesfully, since the computer has to pair two pixels in two images. Because of the extreme simplicication of the image information in this system, the pairing process is much easier. The system only has to pair polygon corners, not single pixels. The polygon paring is done by matching position, color, number of edges and nabour polygons. After that 3D coordinates are assigned to each corner of the polygons using basic photogrametry rules.
By calculating the difference on two sequent 3D polygon-sets, data about the robot movement, such as speed and direction can be extracted. Theese calculations take place in the "Movement Detection"-layer. The output from this layer can be used for calculating odometry / absolute position. In theory the data about the speed could be used as feed-back for the motor controllers.
Initally the plan was to make the robot generate a complete 3D model of the course used for navigation. But it was a too big task for this project. Therefore the top layer is limited to take care of only specific objects found on the RoboCup-course such as white lines, gates, walls etc.
The report for the project can be found here (in danish):
For more information about this project, or a quote for a similar project, please fell free to contact Allan Krogh Jensen.