IMAGE CAPTURE: Neural-network-based cameras speed patterned surface inspection

It has been more than one decade since Guy Paillet, a partner in General Vision (Petaluma, CA, USA; www.general-vision.com), developed a hardware neuron chip called the ZISC (Zero Instruction Set Computer) that used 36 fully hardwired neurons (see “Developers look to neural networks to solve complex imaging tasks,” Vision Systems Design, October 1998).

Since then, General Vision has promoted the concept of neural network computing for vision, offering a number of trainable vision systems based around the technology. One system, the CogniSight4-Iris, a software neural-network implementation based around the Iris smart camera from Matrox Imaging (Dorval, QC, Canada; www.matrox/imaging), for example, has been used to sort fish before they are filleted (see “Vision system expedites fish sorting,” Vision Systems Design, September 2006).

Paillet and Anne Menendez, founder of General Vision, have also pursued the development of a hardware implementation of the device, the CM1K (CogniMem), increasing the number of neurons to 1024 per device. In addition, direct connectivity with a CMOS sensor from Aptina Imaging (San Jose, CA, USA; www.aptina.com) was added to the CM1K as well as I2C communication. Manufactured by OKI (Tokyo, Japan; www.oki.com), this single-chip IC has been developed specifically for applications where images are first trained and then recognized.

General Vision has also developed the CM-V1KU, a reference design that enables camera manufacturers to evaluate and design systems built around an embedded FPGA version of the neural network, an Aptina 9V022/24 CMOS imager, and a USB interface. A Miniature Trainable Vision Sensor (MTVS) using the V1KU board allows up to 100 MTVSs to be connected in parallel to achieve large width surface inspection communicating through a RS-485 multipoint connection.

During the January 2008 Photonics West show, held in San Jose, CA, USA, General Vision discussed a typical defect inspection system application using the company’s neural-network-based MTVS camera. Currently, the company is deploying 36 MTVS systems for patterned surface inspection at a large manufacturing line (see figure).

In an application to detect small defects in a patterned material, an array of 20 MTVS sensors communicates through an RS-485 interface to provide results at 25 frames/s.

In operation, each MTVS stores a codebook representation of up to 1024 learned models of the acceptable aspect of texture. The digitized image is then divided into approximately 1400 tiles, each 16 × 16 pixels, which are matched with this codebook at a rate exceeding 25 frames/s. In this application the captured image consists of a strip of more than 26,000 × 480 pixels inspected at 25 frames/s, without the use of an external computer.

Alternatively, the CogniMem can input a feature vector either from images or other kinds of signals such as sound or sonar. By inputting these reference feature vectors to the neural network, the hidden neuron layer will store a series of subclasses of the feature vector while the output will associate these to a specific category. Recognition is then performed by extracting similar vectors from new images and broadcasting the results to the neural network to see if one or several report a close match.

A theoretically unlimited number of CogniMems can be electrically connected in parallel. This kind of configuration would allow, for example, the closest match of a feature vector to be retrieved, making the system useful in biometric applications.

General Vision is now partnering with GEViCAM (Milpitas, CA, USA; www.gevicam.com) to develop the first GigE camera based on the CogniMem technology.