Sunday, November 10, 2013

RS4 - Robot Update (new features)


RS4 now can detect and read some signs and perform associated actions. In these videos it follows and reads various signs.





How it can read signs using OpenCV


First it must locate a sign and move to it, this is performed by following the blue color around around the signs. I'm having issues with color tracking because the AWB keeps changing the image lightning and at the moment I can't turn it off. The problem is already exposed in the raspberrypi.org forum and they are working in a solution.

When it is close to the sign it performs the next steps to read it:

1. Find image contours


Apply GaussianBlur to get a smooth image an then use Canny to detect edges. Now use this image to find countour using OpenCV method findContours. The Canny output looks like this:



2. Find approximate rectangular contours


All the signs have a black rectangle around them so the next step is to find rectangular shaped contours. This is performed using approxPolyDP method in OpenCV. At this point found polygons are filtered by number of corners (4 corners) and minimum area.



3. Isolate sign and correct perspective

Because the robot moves it will not find perfectly aligned signs. A perspective correction is necessary before try to match the sign with the previous loaded reference images. This is done with the warpPerspective method. Details how to use it here:

In the next image you can see the result of the process, in "B" is showed the corrected image.


4. Binarize and match

After isolate and correct the interest area, the result image is binarized and a  comparison is performed with all the reference images to look for a match. At the moment the system has 8 reference images to compare.
Comparison is performed using a binary XOR function (bitwise_xor). In the next images is showed the comparison result for match and not match, using countNonZero method is possible to detect the match sign.

Match image


 Not match image


Result

This methodology works well and it is fast enough to use it in Raspberry Pi. I've tried known methods like SURF and SIFT but are to slow for real time application using Raspberry Pi.







2 comments:

  1. Your robot is one of my favorite Pi projects. Please keep the updates coming (as and when you get time). Lots of unusual things like the inverted pendulum balancing and the opencv processing.
    Would make an excellent kit if the cost could be kept sensible.

    Also we would like to see it in the MagPi, superb project.

    ReplyDelete
  2. Hello Guy,
    i appreciate your work, and i hope all you can share some code.
    Thank you

    ReplyDelete