Traffic analysis: Traffic monitoring system uses neural networks to recognize license plates
Automatic Number Plate Recognition (ANPR) systems are deployed to enable law enforcement agencies to control and manage traffic on highways. The systems are capable of reading the license plates of vehicles moving at high speeds during day and night and in heavy traffic conditions or where lighting is poor. To identify these license plates, Lector Vision (Madrid, Spain; www.lectorvision.com) has developed an embedded ANPR system dubbed the Traffic Eye that combines off-the shelf and custom hardware and the company's own image processing and pattern recognition software.
The system illuminates traffic scenes using pulsed infrared light while capturing both monochrome images of license plates and color images of scenic overviews using two separate cameras. Software then detects the vehicle plates in the monochrome images and identifies the individual characters on the plates using neural network software. Both the license plate number and the color overview of the scene are then correlated before being transmitted to a control center.
"In the past, many ANPR systems have illuminated license plates using infrared light in the 880nm wavelength range," says Gonzalo Garcia Palacios, Manager of Research and Development at Lector Vision. However, at 940nm, the intensity of sunlight is about 60% of its intensity at 880nm. Hence, Lector Vision chose to reduce the levels of sunlight interference by illuminating traffic scenes with a custom-built array of pulsed LEDs that operate at 940nm.
"The tradeoff when using higher wavelength LEDs is that the sensitivity of the sensor in the camera to light at 940nm is somewhat reduced. To compensate for this, a custom control board in the Traffic Eye pulses the IR LEDs for microsecond intervals that (once reflected from the license plates) can be detected by a monochrome camera. As the scene is illuminated by the pulsed IR light, the controller triggers both cameras simultaneously to enable them to capture both a monochrome and a color image of the traffic scene," Palacios says.
Embedded in the Traffic Eye system is a Blackfly 1920 pixel x 1200 pixel GigE monochrome camera with a Sony Pregius IMX249 CMOS global shutter sensor from Point Grey (Richmond, BC, Canada; www.ptgrey.com). To ensure that visible and IR images can be captured, the camera is fitted with lenses from manufacturers such as Schneider Optics (Bad Kreuznach, Germany; www.schneideroptics.com) and Fujinon (www.fujifilm.com) that feature a color correction and broadband coating. Images from this camera are then transferred at 25fps to an embedded vision controller. A second Blackfly 1920 pixel x 1200 pixel GigE color camera, also from Point Grey, equipped with a Sony IMX249 CMOS sensor, is also embedded in the system to capture an overview of the scene at 25fps.
To determine the characters on the license plates in the image, the monochrome image is analyzed in the embedded vision system controller. To do so, custom-built software first searches for rectangular regions of interest (ROIs) in the image where a license plate is likely to be present. "By capturing license plates from just one image an accurate license plate reading is obtained, and analyzing up to 10 consecutive images, maximum performance is reached." says Palacios.
To identify the individual characters on each of these license plates, Lector Vision uses neural network software. To do so requires presenting the system with approximately 10,000 images of license plates whose alphanumeric characters are then manually identified. This is accomplished by first deploying the system and capturing thousands of separate examples that are then manually identified and stored in the system's database. Once the database has been created, the neural network is then trained, and any unknown license plate image can be then captured and identified.
Once identified, the license plate number and the image of the traffic scene captured by the color camera and an optional GPS time stamp are sent to a control center over cable, optical fiber, GPRS or wireless 3G. In doing so, highway agencies that monitor traffic flow and enforce red light violations can view the images to identify a vehicle and examine the image of the automobile and its occupants at the location where the license plate of the vehicle was captured.
According to Palacios, over 500 Traffic Eye systems have been installed since the system was introduced in 2013 where they have proved capable of reading number plates of vehicles travelling at speeds that exceed 200km/h. Traffic Eye systems have been installed in hundreds of locations in Spain, Portugal, Poland, Slovakia, Chile, Colombia, Peru and Mexico.