Comparing Sensors and Cameras

Andrew Wilson, Editor

It has been more than three years since the European Machine Vision Association (EMVA) proposed a standard method for the characterization and presentation of technical specifications for image sensors and cameras. Known as EMVA-1288, the standard aims to benefit sensor and camera vendors and their customers by providing data that allows such products to be rapidly evaluated and compared.

Despite these efforts, however, few manufacturers—notably Awaiba (Madeira, Portugal; www.awaiba.com), PCO Imaging (Kelheim, Germany; www.pco.de), and Basler (Ahrensburg, Germany; www.baslerweb.com)—have produced data sheets that comply with the standard. One of the reasons for this, perhaps, is the time needed to develop an in-house test system capable of meeting the requirements of the standard.

Even when this type of system is developed, the measured results must be computed using mathematical models to generate the required results. Test systems using the EMVA-1288 standard must therefore compute values such as absolute sensitivity, dynamic range, and quantum efficiency based on measured parameters. Companies such as Basler have developed their own systems for in-house use and have qualified many of their cameras accordingly. To date, no off-the-shelf EMVA-1288 sensor/camera test system has been commercially available.

Automated test

However, at Photonics West 2009, held in San Jose, CA, in January, Sarnoff (Princeton, NJ, USA; www.sarnoff.com) introduced its Camera Commander, a sensor and camera test system that fully automates the measurement of such devices, reducing the time to produce EMVA-1288 compatible test data to fewer than five minutes per device.

“At present,” says Jongin George Kim, of the company’s Imaging Products Division, “the EMVA-1288 standard only covers monochrome digital area-scan cameras with linear photo response characteristics. However, it is expected that the standard will soon encompass linescan and color cameras.”

From a sensor or camera manufacturer’s perspective, four of the most important measurements within the EMVA-1288 specification are the absolute quantum efficiency, signal-to-noise ratio (SNR), and the nonuniformity and nonlinearity of the device under test. For these parameters to be accurately measured, the system must perform a number of different tests.

To determine how a specific wavelength can affect the total quantum efficiency, for example, a series of measurements must be made using monochrome light of different wavelengths. For each wavelength, the light’s intensity and the exposure time must be adjusted so that the same amount of photons arrive at the sensor or camera during the exposure time. A bright measurement is then taken and the mean gray value of the image computed. After this measurement is taken, the camera or light source must be capped, a dark measurement taken, and the mean gray value computed. From this data, the quantum efficiency of the sensor or camera can be computed over a number of different wavelengths.

System design

“As can be seen,” says Kim, “just one single measurement such as this can be a very time-consuming process if performed manually. However, using the Camera Commander system, these functions are all automated under software control and can perform a complete EMVA-1288 test specification very quickly.” To accomplish this, Kim and his colleagues developed the PC-based system using off-the-shelf optics, light sources, and frame grabbers and proprietary image analysis and presentation software (see Fig. 1).

FIGURE 1. Sarnoff’s Camera Commander fully automates the measurement of sensors and cameras, reducing the time to produce EMVA-1288 compatible test data to fewer than five minutes.

To produce the uniform diffuse scattering of light needed in the system, the Camera Commander uses an integrating sphere from Sphere Optics (Concord, NH, USA; www.sphereoptics.com). By using this integrating sphere, light rays from a white NIST-calibrated photodiode that are incident on any point on the inner surface are distributed equally by multiple scattering to all other points within the sphere and the effects of the original direction of the light is minimized.

To maintain a consistent light output to the photodiode, the device is driven by a Picoammeter from Keithley Instruments (Cleveland, OH, USA; www.keithley.com) under RS-232 control. This allows voltage and current to the device to be measured and the system to be automatically calibrated.

As light emerges from the 2-in. diameter of the integrating sphere, it impinges on the sensor or camera under test. To be compliant with the EMVA-1288 standard, these devices must be placed 16 in. away from the light source. In the Camera Commander system, both the integrating sphere and camera under test are mounted on an optical bench to ensure extremely accurate alignment of both the light source and device under test.

Camera interfaces

“Because the system must support a range of cameras with different interfaces,” says Kim, “a number of interface boards are required in the host PC. These include an X64 Xcelera-CL PX4 Full PCI Express board from DALSA (Waterloo, ON, Canada; www.dalsa.com) to support Base, Medium, and Full Camera Link cameras and a 1394a/b interface board from Unibrain (San Ramon; CA, USA; www.unibrain.com) to support FireWire cameras.”

The system incorporates the GigE driver from JAI (Copenhagen, Denmark; www.jai.com) while USB 2.0-based cameras are supported through the host computer’s on-board USB interface. Using these interfaces, camera parameters such as exposure time and gain can automatically be configured using the Camera Commander’s system software.

To measure parameters such as the quantum efficiency of the camera at various wavelengths, the integrating sphere is also equipped with a shutter from Spectral Products (Putnam, CT, USA; www.spectralproducts.com) that allows dark current to be measured. To obtain a plot of quantum efficiency at various spectral frequencies, this sphere is also equipped with a series of six bandpass filters mounted on a filter wheel that range from 400 to 750 nm. Under computer control over an RS-232 interface, these filters can be automatically positioned so that measurements of quantum efficiency at various wavelengths can be computed.

In the design of the Camera Commander system, Kim and his colleagues have paid special attention to the user interface (see Fig. 2). The results shown are for Sarnoff’s own CAM1M100 digital camera that uses a 1024 × 1024 CCD sensor with 16 × 16-μm pixels. As well as allowing the user options to set specific functions within the system for tailored image measurement, the software allows users unfamiliar with testing sensors and cameras a one-click option that performs all the tests specified by the EMVA-1288 standard.

“While some camera manufacturers will want to see parameters such as quantum efficiency, system gain, system nonuniformity, and SNR performed at various frequencies,” says Kim, “many will simply require these figures to be computed at a single wavelength near peak response.”

Using such systems will make it easier for vendors of sensors and cameras—and more important, for those who specify such devices—to accurately compare devices from different manufacturers. In this way, those who previously struggled through disparate data sheets that may or may not have reflected the actual performance of sensor and camera products will be amply rewarded.