PDF version of this entire document

Hardware and Performance

The aforementioned screenshots and explanations sometimes rely on desktop emulation; in practice, I wish to investigate performance on real hardware in real-world situations. After many hours that involved studying a lot of options (both device and desktop end), then editing files etc. I managed to successfully install my program on an Android 4.0 tablet. For the basic edge detection and analysis I can get about 4 frames per second and the hardware specifications include a 10 screen (VGA), 1GHz CPU, and 1GB of on-board RAM ⁶. Testing over USB is trivial and fast once particular workarounds are mastered, reducing lag. Regarding the hardware I currently test and work with, it is important to note that it is supposed to take account and align with what is cheaply available on the market right now. Cutting edge would be unrealistic and overly optimistic.

When I proceeded to doing profiling for feasibility I found some caveats and bottlenecks. For a window size of 402px by 512px (an inner window, leaving margins out) the performance is a little better than emulation on x86 with half a gigabyte of RAM. The hardware here is not something advanced like Tegra (those are expensive) and unlike some Sony-made ICS tablets that come with high-resolution cameras at the front and the back, this one uses a resolution which can be managed by real-time image processors. If I can only deal with a dozen frames per second, this may still be enough to get prediction of objects' motion on the road.

At the earlier staged I applied some edge detectors and then tried to identify a silhouetted road part, which could then be stripped apart to avoid performing computation on irrelevant parts like the sky. At earlier stages I also tried to apply template matching (cars) using one of 6 methods that I have, but it slows down the pace of processing to sub-1 (FPS) levels, so I needed to decrease the resolution for this part (coarse or coarse-to-fine). At that moment the program was leaking some memory, so I was still debugging it.

I are sure one can do better than 5 frames per second, but while developing and debugging the problem I do not worry about performance too much, at least not yet. For standard video sequences on this ARM-based device I can sometimes get about 8 frames per second (raw and idle). Maybe it is a hardware limitation, related perhaps to throughput or lack of hardware acceleration (some x86 chipsets have MPEG acceleration on-board). Tagra tablets (they are all rather expensive) have very smooth video in comparison. They are HD as well. Trying to run the same program on them would be an interesting exercise.