LEADING EDGE VIEWS - Machine Vision Moves into Traffic: Part II
Choose optimal software functions to meet the application challenges of ITS tools
In developing machine-vision systems, designers can leverage the power of open source andcommercially available software packages such as OpenCV, MIL (Matrox Imaging Library), VisionPro, eVision, Sherlock, and Voyant Vision. Because these software packages contain numerous general-purpose algorithms, they can also be used in other types of applications such as intelligent transportation systems (ITS). For example, frame averaging can eliminate rapidly changing features from a scene, and frame differencing can detect motion within a scene.
One of the most common image-processing operations is the measurement of the gray-level average -- the spread of the distribution of pixel intensities within a specific region of the image. However, performing a gray-level average is rarely used in ITS applications because light levels in the scene may change. Rather, to determine if (and by how much) pixel values vary in a specific region of an image, the standard deviation of pixel values is more often used. As lighting changes in a scene, the standard deviation between pixel values is relatively low, but as soon as an object enters the scene, there will be a marked deviation.
To separate the foreground of an image from the background to isolate objects of interest, image thresholding can be employed. In transportation applications, however, it is rare to find a case where an object can be imaged against a uniform background, and often the object to be imaged is not of a standard intensity. However, thresholding can be used after image subtraction between two images to highlight the differences between them. Color-based thresholding can deliver better performance yet, if a color difference between the two images exists.
Convolution-based and nonlinear filters are also commonly used in transportation applications. Smoothing can be used to remove noise, edge enhancement to differentiate between two images, and Gaussian blurring or smoothing filters to remove the effects of dirt, snow, and dust.
Because conditions are so controlled in most machine-vision applications, exhaustive testing of software algorithms on numerous images is frequently unnecessary. Often, it is possible to specify which algorithms will be effective from a few test cases. However, in ITS applications, generalizing which algorithms will be effective from a few test cases and then developing software is not recommended. To ensure that systems will be successful, engineers must employ the most effective algorithms and test them on large image databases to ensure they are general-purpose enough to be used.
Morphological operators
Morphological operators are widely used in many ITS applications since they process objects in an image based on their shape characteristics. These operators can be categorized into seven types (see table); the two most commonly used are erosion and dilation. These can be used in conjunction with other algorithms, for example, to identify the location of a wheel on a pickup truck (see Fig. 1). After performing a thresholding operation to locate the wheel, the image will still contain a group of dark pixels that may make the image difficult to process further. In addition, there may be spurious pixels in the background that represent noise. By using erosion and dilation, both the spurious pixels in the center of the image of the wheel and the noise from the image can be eliminated.
In transportation applications, images can frequently be distorted by the position of a camera as it captures an image at an oblique angle, or by the use of fisheye or panoramic lenses. If well understood, such distortion can be compensated in software using geometric transformations.
Point Grey’s LadyBug camera is an example of the type of camera used in ITS applications. This camera features six Sony CCD image sensors, with five of the CCDs positioned in a horizontal ring and one positioned vertically. Together, these perform 360° visual coverage of a scene (see Fig. 2). Here, geometric transformations can be applied to all the images as well as blending and stitching prior to further image processing.
Calibrating cameras
One important task performed by developers of machine-vision systems is the calibration of the camera or cameras used in a system. Calibration performed using calibration targets, such as checkerboards, enables designers to determine how the geometric and optical characteristics of the camera must be corrected to capture an accurate rendition of an image.
In ITS applications, the area to be imaged by the camera often is very large, rendering it impossible to build or lay out a precise target from which the camera can be calibrated. One solution is to use the geometry of specific features of the infrastructure that are found in the scene to calibrate the system. In a rail application, for instance, the geometry of rails in an image might be used for either two- or three-dimensional camera calibration.
In Part III of this series, Dr. Ned Lecky from Voyant Vision will offer a detailed look at feature extraction, pattern matching, and OCR used for ITS applications.