Image sensors build on the brilliance of Bayer

Andrew Wilson, Editor

Invented in 1976 by Bryce Bayer, the Bayer filter is perhaps the most well-known color filter arrays currently in use. Elegant in its simplicity, the well–known filter uses an array of RGB filters placed on a sensor array to allow image sensors to create color images. By using twice as many green as red or blue filter elements, the filter array was designed to more accurately emulate the physiology of the human eye since the eye is more sensitive to green wavelengths (Figure 1a).

Click to EnlargeFigure 1: (a) While the well-known Bayer color filter array uses a 2 x 2 array of repeatable RGB filters, variations such as (b) Truesense's Truesense CFA adds panchromatic (or clear - C) pixels, while (c) Aptina eliminates the green filter using just R, B and clear (C) pixels interpreting the G by subtracting R and B values from C. (d) In its latest X-Trans CMOS imagers, Fujifilm has chosen to increase the randomness of the arrangement of pixels in 6 x 6 sets while (e) Panasonic has patented a method that replaces these color filters with inorganic color splitters.

Despite its popularity, the Bayer filter pattern represents just one method of producing color images. Indeed, in the near four decades since its invention, many other different filter patters have been proposed, each with their distinct advantages and disadvantages.

One variation on the Bayer pattern is the Truesense color filter pattern from Truesense Imaging (Rochester, NY; USA; www.truesenseimaging.com) that adds panchromatic (or clear - C) pixels to the RGB pixels (Figure 1b). This increase in light sensitivity is traded for the resultant lower resolution of interpolated color pixels resolved using demosaicing algorithms.

Truesense is not the only company that has spurred renewed interest in the use of panchromatic pixels. Aptina (San Jose, CA; www.aptina.com), for example has recently introduced what the company calls a Clarity+ color filter array (CFA) pattern that uses a combination of red, blue and clear (C) pixels (Figure 1c). Unlike the Truesense CFA, however, no green filters are used. Instead the green value is determined through a subtraction of R and B from C. According to Aptina, using the same number of C pixels as other 50% C schemes, the same SNR improvement can be achieved but because G is determined through a subtraction, there is a tendency to amplify noise in the conversion to the standard RGB representation. This, the company says, is compensated for by image processing algorithms that are part of the Clarity+ technology (http://bit.ly/LGXEUj).

Rather than incorporate clear pixels in its X-Trans CMOS imagers, Fujifilm (Tokyo, Japan; www.fujifilm.com) has taken a different approach inspired by silver halide film. Since the repeating arrangement of the 2 x 2 pixel sets of the Bayer pattern tends to generate Moiré and false colors when imaging stripes and other regular patterns, Fujifilm has chosen to increase the randomness of the arrangement of pixels in 6 x 6 sets to reduce the occurrence of Moiré (Figure 1d). According to the company, the presence of an R, G, and B pixel in every vertical and horizontal pixel series minimizes the generation of false colors and delivers higher color fidelity (http://bit.ly/LvcWL9).

Because conventional color image sensors use a Bayer or variation of Bayer arrays, incoming light is naturally filtered by the RGB filters before it reaches the detector array. Rather than take this approach, Panasonic (Osaka, Japan; www.panasonic.com) has patented a method that replaces these color filters with inorganic color splitters to separate the incoming light (Figure 1e). In this approach, the structure separates light into color pixels of white + red, white - red, white + blue, and white - blue and, using image processing techniques are then computed as color images (http://bit.ly/1hG1TKp).

While many of today's image sensors incorporate silicon detectors, many researchers are looking to replace these with light sensitive organic compounds that either use conventional silicon-based read out circuits or replace both the detector and read out circuits with organic compounds (see "Organic imagers challenge silicon-based detectors," Vision Systems Design, February 2014, p. 13).

Notable developments in this area include image sensors from the Technical University of Munich (TUM, Munich, Germany; www.tum.de), Fujifilm and Panasonic, IMEC (Leuven, Belgium; www2.imec.be) and the Holst Centre (Eindhoven, The Netherlands; www.holstcentre.com) and ISORG (Grenoble, France; www.isorg.fr) and Plastic Logic (Cambridge, England; www.plasticlogic.com).

Many of the latest CFA and image sensor designs are at present targeted towards specialized consumer, security and x-ray applications. However, such developments – or variations of them – will no doubt eventually be incorporated into cameras developed for industrial machine vision and image processing applications.