Airborne camera acquires IR and visible spectrum images

Dual-spectrum-board camera combines visible CCD and IR detector to capture images in visible and mid-IR bands or both simultaneously.

By Jeff Child, Contributing Editor

The capability to view nearly real-time digital images in both the infrared (IR) and visible spectrums simultaneously is crucial for military commanders when analyzing battlefields and other important combat scenes. Accomplishing this in a single vision system requires advanced camera technology integrated with high-throughput computing capability. Engineers at Recon-Optical Inc. (ROI; Riverside, CA) took on this design challenge to craft an advanced dual-spectral-band camera.

FIGURE 1. Infrared-image data captured in real time are sent to the Image Processing Unit, an enclosure chassis separate from the Stabilized Imaging Unit containing the IR and visible sensor modules. Image data from both the visible and IR sensors are fed directly into the RACE++ switch fabric to a pair of Mercury MCJ6 VME processing boards, each containing multiple PowerPC Altivec processors. These processors crunch through the scene-based nonuniformity-correction scheme needed to produce the necessary output images. They share work as necessary between boards and between on-board processors via the 533-Mbyte/s bidirectional RACE++ links.

At the core of the IR system design is the complex nonuniformity-correction scheme that is required to process streaming IR image data. To process and display these streaming images in nearly real time, the IR system needs to process four 4 million-pixel frames/s. Moreover, the system has to process all the streaming pixels in a frame and perform a histogram function on the information and then apply it to the next frame. This IR-image processing is conducted in parallel with the management of imagery captured by a 25 million-pixel visible-spectrum sensor also operating at 4 frames/s.

Handling that high level of image-processing performance requires more than leading-edge computational capability. Equally important is the ability to move streaming data in and out of the processors as fast as possible. Based on these challenges, ROI engineers chose to integrate PowerPC Altivec-based processor boards from Mercury Computer Systems (Chelmsford, MA) that are linked via the company's RACE++ switch-fabric interconnect technology.

Two versions of the SIU were built for different mission types. The medium-altitude SIU (SIU/M) provides an 18-in. focal length, whereas the high-altitude SIU (SIU/H) features an 84-in. focal length. Electrical and data interfaces between the SIU/M and SIU/H are identical. The IPU and PCU are common to both versions of the SIU.

The road to dual-spectrum-band capability began with ROI's development of its CA-260 camera. This camera offered digital-imaging capability for the visible spectrum. The company also developed the CA-265 IR camera using a large-area IR detector.

FIGURE 2. Stabilized Imaging Unit houses both the IR and visible sensor modules, camera electronics, and power supplies for the Recon-Optical airborne dual-spectrum-band camera. This unit also runs the company's patented stabilization software on a digital-signal processor, which enables the camera to remain stable relative to the aircraft's motion.

Both these visible and IR detectors used patented on-chip motion compensation that electronically removes the affects of an aircraft's forward motion that would normally cause captured images to smear. Starting with these large-area IR and visible spectrum detectors, ROI engineers developed an advanced sensor system that uses both detectors simultaneously. The resulting dual-spectrum-band camera combines the performances of a 25-Mpixel visible CCD sensor and an IR detector to capture digital-framing imagery in the visible band and in the mid-wavelength IR band, or both spectral bands simultaneously. The camera provides a horizon-to-horizon field of regard, a wide single-frame field of view, and stereo imagery for enhanced imaging analysis and precision targeting.

The camera works as a step-frame sensor, taking a sequence of adjacent frames in the across-line-of-flight direction. The number of frames taken in this direction is a function of time, which is determined by the velocity-to-height ratio of the mission and the desired maximum depression angle in the framing sequence.

The performance capabilities of the SIU/M and SIU/H assemblies complement each another. The performance of the SIU/M satisfies the entire operating envelope for a typical medium-altitude mission and also extends into high-altitude envelopes. Meanwhile, the SIU/H provides high resolution at long stand-off ranges and covers much of the medium-altitude operating envelope.

Overlap between image frames taken along the flight direction ranges from 0% to 100% and is user-selectable. When frames are overlapped at 55%, the camera system produces stereo images. In these images, objects such as towers and trees become extremely vivid and appear to jump out at the viewer. A 10% overlap is the minimum typically used for monochrome coverage. According to Steve Wyatt, ROI director of operations, the step-framing technique offers excellent geometric fidelity in each frame beyond that achievable by linear systems, especially under tactical maneuvering platform conditions. Advantages of step framing inherent in the dual-spectrum band include wide area coverage, high resolution, and geometric fidelity.

The SIU transfers full-resolution image data to the IPU. Communications between the IPU and SIU are done over RS-485 interfaces. After the data are received on the Mercury RINOJ input boards in the IPU, data are fed directly over the RACE++ switch fabric to a pair of Mercury MCJ6 VME multicomputer boards. Each board contains multiple PowerPC Altivec processors. Those processors crunch through the scene-based nonuniformity-correction scheme needed to produce the necessary output images. They share work as necessary between the boards and between the on-board processors via bidirectional RACE++ links. Wyatt says, "The image data are processed at 4 frames/s through both the visible and IR channels in real time. That's a tremendous amount of data to move and process quickly."

The RACE++ technology increases the aggregate bandwidth of the RACE crossbars from 480 Mbytes/s to 1 Gbyte/s. The crossbar ASIC within the RACE++ interconnect fabric is an eight-port switch. Like its RACE predecessor, the RACE++ architecture allows data to choose in real time the least congested data path between source and destination. This capability, called adaptive routing, is executed entirely in hardware, without the need for system or user software overhead. Whereas the RACE architecture allowed two (out of six) ports to do adaptive routing, the RACE++ architecture uses all eight ports to implement adaptive routing.

Providing system control and management is a 178-DVM, VME single-board computer from DY-4 Systems (Kanata, Ont., Canada). This board loads program software from flash memory onto the MCJ6 multicomputer boards. Because the Mercury boards share data over RACE++, the DY-4 CPU gains more bandwidth on the VME bus.

The DY-4 CPU board performs system-management tasks by running the VxWorks real-time operating system from Wind River Systems (Alameda, CA). The image-processing software running on the MCJ6 multicomputer boards was developed by ROI using C++. Aside from command and management communications, the VME bus also transmits thumbnail imagery (decimated images) from the MCJ6 boards to the 178DMV board for display on a cockpit video monitor. All these boards are rugged, conduction-cooled products. They are designed for battlefield conditions and high altitudes in which the dual-spectrum-band camera will be used. In such harsh environments, extreme temperatures, humidity, dirt, dust, vibration, and low air pressures can degrade reconnaissance imagery.

Whereas military systems are known for their long design cycles, ROI engineers made use of commercial off-the-shelf products and strategies to shrink development time. They were able to move technology developed for nonruggedized boards and quickly transition it to conduction-cooled boards without modifications. According to Vince Mancuso, vice president and general manager of Mercury's government electronics group, "Recon-Optical ordered the conduction-cooled versions of our MCJ6 boards so that they could use the existing code that was already developed on the commercial versions. While the conduction-cooled versions differ in form factor and weight from the commercial versions, they're otherwise electronically and mechanically the same."