TRANSPORTATION IMAGING: License plate recognition software targets embedded systems
Automatic license plate recognition is used in a number of security and surveillance systems to provide information as a means of electronic toll collection or to track the movement of specific vehicles.
In many of these systems, remotely located cameras are used to capture the images of the license plates, transmitting the images to a host computer for image enhancement and optical character recognition (OCR; see figure). This information can then be used in conjunction with existing data to provide information as to whether, for example, a vehicle may have been stolen.
With the advent of FPGA-based smart cameras, it is now possible to perform image enhancement and OCR techniques directly within the camera. This offloads the task from the host PC, increases the processing speed, and allows the camera to transmit the license plate information over the camera-to-host interface.
In developing its license plate recognition software, Irida Labs (Patras, Greece) recognized the need to allow smart camera developers to embed this code directly within the FPGA of the camera. According to Nikos Fragoulis, vice president of technology, the company's software can be embedded in any system that employs either Altera or Xilinx FPGAs, enabling license plate detection in 1280 × 720-pixel images to be performed in approximately 10 ms. This equates to a typical 90 frames/s—far faster than could be achieved using a standard single-core PC.
Fragoulis and his colleagues compiled the VHDL code used for license plate recognition and transferred the code over a JTAG interface to the board's Xilinx Spartan III FPGA. According to Fragoulis, this code occupies about 30% of the available logic on the device.
To perform OCR on images as fast as 90 frames/s, the embedded software within the FPGA first performs an adaptive thresholding that accommodates changing lighting conditions in the image. The resultant binary image is then segmented using connected component analysis to identify connected regions in the binary images. Filters are then applied to the thousands of candidate regions identified. One of these filters concerns the horizontal scanning of the region in order to determine whether a typical rectangular aspect ratio that would represent a typical license plate exists in the image. If such a potential candidate does exist, OCR software is applied to these isolated images to determine the characters of each license plate present.
Irida Labs has teamed with Framos (Pullach, Germany) to deploy the software on the company's VFx-P scalable performance camera platform. This open hardware concept allows the developer to choose from a number of sensor, processing, and interface boards to build an embedded camera based on only those components required.
As part of this concept, the company's FPGA-based processing board, the VFx-P, employs a Xilinx Spartan 6 and 128-Mbyte DDR2-RAM, and can be programmed in VHDL over the board's JTAG interface. According to Dietmar Scharf, senior software manager with Framos, Irida Labs' software has already been successfully demonstrated on a camera that employs the VFx-P processing board, and a camera equipped with the software will be made available from Framos in mid-2012.
For its part, Irida Labs plans to make the software available to other FPGA-based camera vendors. According to Irida's Fragoulis, the software may be sold as part of a license agreement or delivered in a camera as a complete end-user product.
Correction
In the October 2011 article "Filters Factor into Optical Imaging" by Stuart Singer, some of the images were not placed in the correct order. You can see the corrected version of the article with artwork online at http://bit.ly/mT6zyU. Vision Systems Design regrets the error.