So Far...

I've been out sick for a little more than a week. I am almost completely well again and will resume researching.

I will attempt to summarize everything that has happened so far with regards to my work.

Last Tuesday, January 10, my primary contact came back to work after being sick. He showed me how to interact with the robot that I will be working with.

Introducing Robart III

The robot's name is Robart III and is named after Dr. Bart Everett. Here is some background on Robart III:

http://www.nosc.mil/robots/land/robart/robart.html#Robart3

I am very fortunate to be working with Robart III to detect soda cans. He is allegedly one of the most sophisticated mobile robots in the world for use as a research platform. He has been featured numerous times on the Learning, History, and Discovery Channels and is ranked #16 out of Wired Magazine's list of the top 50 greatest robots ever:

http://www.wired.com/wired/archive/14.01/robots.html?pg=2

Robart's Gun

Although his gun looks like a m134 “minigun” it is actually just a 6-shot air rifle that shoots little darts or rubber pellets. This will make testing my soda can detection project more exciting! Here is some footage of Robart III shooting soda cans with his rifle in a very controlled environment. This was done before I arrived. In this video, the soda cans are set at a fixed distance, with no background clutter, in a plywood box which would minimize the effects of ambient lighting on the detection process. I believe that a simple cross-correlation is taken to determine if soda cans are being detected. My implementation will hopefully be much more robust and will integrate well with Dlagnekov's LPR/MMR (License Plate Recognition / Make Model Recognition) code.

http://www.nosc.mil/robots/movies/rob3coke.mpg

Here is a video of Robart shooting balloons.

http://www.nosc.mil/robots/movies/robart3.mpg

Robart's Vision Computer

Robart has several computers to perform different functions. The computer I will be working with is his vision computer, which is stored in his head. His vision computer consists of a Pentium 3 CPU mounted on a stackable computer where extra peripherals can be added on by literally stacking them on top of each other. This allows the computer to be expanded upward without taking up any extra space around it. To log into the computer, I simply run a standard CAT5 network cable from Robart to my laptop and use SSH.

Status of Louka's Code on Robart

Last quarter, Stephan and Kevin ran Dlagnekov's LPR/MMR software on Robart. They took a stripped-down distribution of Linux and stripped it down even more so it could run on Flash Memory (the storage medium for Robart) without burning out the cards from excessive read/write accesses. This was then installed on Robart's vision computer. Stephan and Kevin also developed code to take video footage from Robart, run Dlagnekov's LPR/MMR code on it, and create a video file with the detected license plate locations outlined.

Stephan and Kevin worked mostly on their own computers and generated a binary file to run on Robart. We uploaded this file along with the appropriate libraries to Robart and attempted to run Dlagnekov's LPR/MMR code with Stephan and Kevin's code. Unfortunately, we were unsuccessful due to the fact that Stephan compiled the library file with a different version of g++ than that which was installed on Robart.

When I got home, I emailed Robart's output to Stephan and requested that he send his source code to us so we could recompile it. Stephan was very helpful and emailed the code to me, along with the successful output from running the code on his machine.

While I was at home, I installed OpenCV and ffmpeg on my laptop in hopes that I could run Dlagnekov's code and Stephan and Kevin's library file on my laptop. When I came in to work, I found that I was missing some libraries for OpenCV and ffmpeg so we copied them from his computer onto my laptop.

So far, we have not been successful in running Dlagnekov's LPR/MMR code on Robart or my laptop. In order to move forward with my project, I have downloaded the USBVISION library to my laptop. Once this works on either my computer or his, we can then transfer our USBVISION setup to Robart. This will enable me to begin building my training set of soda cans and acquire footage for testing.

The Applicability of Dlagnekov's Thesis to Soda Can Detection

In between the software compatibility problems, I have been continuing to read Dlagnekov's Master's thesis on LPR/MMR. I am almost finished with it and have taken copious notes. Throughout my reading, I have noticed many aspects of Dlagnekov's project that are specific to license plates and cars but may not work well for soda cans.

One thing I noticed is that Dlagnekov builds a training set of license plates taken at 3 separate distances. Since the cars are recognized from the rear as they are driving away, there is a good chance that the license plate will match at least one of these three sets of images. In my soda can detection project, the soda cans will be stationary. If Robart zooms in on a soda can, then the soda can could be matched from a training set of 3 different soda can sizes. Herein lies the problem: how can Robart zoom in on a soda can without detecting its presence in the first place? Although I have so far come up with more problems than viable solutions to them, this is at least helping me to understand the present framework more thoroughly. By anticipating these problems early on, I can begin to explore the possible role that other research findings, such as SIFT with its scale and rotation invariance, can take to help with soda can detection.