Foveal vision imparts intelligence to camera designs

Foveal vision imparts intelligence to camera designs

Whereas conventional image-processing systems collect data with uniform acuity, virtually all advanced biological vision systems perceive images in a multiresolution fashion, where visual acuity varies by several orders of magnitude within the field of view (FOV). When supported by lower peripheral acuity, the region of the retina with high acuity, called the fovea, can acquire more relevant information in certain vision applications than a uniform acuity sensor with the same number of photosensors. Consequently, foveal vision requires a tighter integration of visual processing and behaviors than uniform acuity vision. Amherst Systems (Buffalo, NY) is developing a hierarchical-foveal-machine-vision (HFMV) system that achieves the throughput benefits of foveal vision while retaining the commercial feasibility of uniform-acuity machine vision.

Two types of HFMV systems are being developed: fixed lattice and reconfigurable. Fixed-lattice HFMV uses an imager whose distribution of variably sized photoreceptors (called resolution elements, or rexels) is scaleable and rectilinear (see figure). This imager was developed in collaboration with the Department of Electrical and Computer Engineering at the State University of New York (Buffalo, NY). Fixed-lattice HFMV derives rexel values optically to yield fast frame rates and large-format imagers, but requires a pointing mechanism to move the optical axis. Because of this, Amherst Systems has collaborated with Moog (East Aurora, NY) in the development of a prototype 2-DOF servo actuator that yields greater than 1000°/s pointing for 100-gram miniature cameras.

To match spatial resolution, temporal resolution, and FOV requirements as they change during target searching, detection, interrogation, and tracking, Amherst has also developed a reconfigurable HFMV multiresolution imager. In this device, the distribution of rexels is programmable, and the system derives rexel values by averaging pixels with on-chip circuitry. In operation, video-processing algorithms control the imager configuration and implement gazing electronically by situating the fovea at different locations within the imager`s FOV.

Based on this technology, Amherst Systems has joined with the NASA Jet Propulsion Laboratory (JPL; Pasadena, CA) in the development of a reconfigurable HFMV camera. Using an active-pixel-array imager with programmable on-chip pixel-to-rexel averaging, the camera`s imager also features on-chip circuitry for fixed-pattern and sampling-noise removal. Because the camera is programmable, reducing the video window size and resolution proportionally increases the frame rate. The camera is now available as an integral part of a closed-loop HFMV system.

Because reconfigurable HFMV adapts its imaging parameters on a frame-by-frame basis to satisfy time-varying requirements, it can be used to add intelligence to machine-vision systems, particularly those that operate in poorly controlled environments or those that, at present, use mechanically articulated cameras in their design.