In many military and scientific image-processing applications, it is necessary to examine several images from different parts of the frequency spectrum. This is especially useful in applications such as remote sensing, where multiresolution satellite data can provide the spatial and spectral data required for vegetation classification. In military applications, the analysis of both IR and visible data can allow potential targets to be classified more easily. To present the data to the user, a number of companies, including Equinox (New York, NY, USA; www.equinoxsensors.com) and Pyramid Vision (Princeton, NJ, USA; www.pyramidvision.com), are building systems that permit data from visible and IR systems to be fused and displayed in a single image.
According to Christopher Eveland, director of engineering at Equinox, this task is not trivial. “Often,” he says, “images from visible and IR cameras have different pixel sizes. As well, the lenses used in such systems will exhibit different degrees of barrel and pincushion distortion.” When fusing images from visible and IR cameras, the different pixel sizes must be scaled and any distortion corrected.
Both Equinox and Pyramid Vision offer products that merge data from different imaging modalities such as visible and IR. While the Equinox Stratix-based FPGA allows images from different FPAs to be combined and displayed, Pyramid Vision’s Acadia PCI board adds image stabilization and image interpretation to the fused data.
To meet these demands, Equinox has developed the Dual Video Processor DVP-4000-a 3 × 3-in. video processor. Performing image fusion, coregistration correcting for nonlinear optical effects, and automatic gain control on an FPGA, the DVP-4000 consumes 1.5 W. Using image fusion alone, the FPGA consumes less than 1 W.
In addition, the board’s Stratix-based FPGA allows images from different focal-plane-array resolutions to be combined. “To capture images from the DVP-4000, the board’s Camera Link output allows off-the-shelf frame grabbers such Phoenix D-36 from Active Silicon (Uxbridge, UK; www.activesilicon.com) to digitize fused images into a PC. Images can be displayed in real time with the on-board graphics controller.
Equinox has used the device in its Equinox Ace camera, which the company claims to be the world’s first dual-mode facial-recognition system. In the design of the camera, Equinox has incorporated a microbolometer and a low-light-level CCD, both focused on the same subject area using a beamsplitter. “By combining visible and IR images,” says Eveland, “first match error is reduced by up to 85% compared with visible-only facial-recognition systems.”
“In many image-fusion applications,” says Donald Channin, business development manager at Sarnoff Corporation (Princeton, NJ, USA; www.sarnoff.com), “it is necessary to perform additional functions such as image mosaicing from successive image frames, image stabilization, and image interpretation.” To accomplish this, Sarnoff developed a custom ASIC, Acadia, that builds on the concept of pyramid processing developed by Peter Burt, director of the Sarnoff Vision Technology Laboratory. This allows weights to be assigned to each individual pixel in both IR and visible images, depending on which contains the most information. “By weighting visible and IR images depending on their content, more information about the combined content is revealed,” says Channin.
To allow system integrators to evaluate the Acadia technology, Sarnoff has spun off Pyramid Vision to market a PCI-based image-processing board. Using multiple broadcast-compatible inputs such as NTSC, PAL, and SVHS formats, images from different modalities can be captured and displayed. As well as combining images, the board’s on-board ASIC eliminates jitter and shaky imagery from cameras in real time with input frame stabilization of up to 1/10 pixel. And, Pyramid offers the board in a rugged MIL-SPEC 810 version designed to be installed in aircraft and ground vehicles. Ruggedized systems that use both the Acadia PCI vision board and the Acadia system are also offered by Pyramid Vision’s partner MicroDISK (Yardley, PA, USA).