Vision system excels in inspection of colored auto fuses

Color-based machine-vision system detects and identifies the proper position of automobile fuses.

During the past decade, machine-vision systems have performed as cost-effective inspection tools using gray-scale hardware and software algorithms. More recently, the development of color-based machine-vision systems has expanded the power and diversity of these robust inspection tools. Color provides useful information for object classification, sorting, inspection, verification, and anomaly detection, and color-based systems have automated these operations. However, the successful implementation of color systems requires an expanded level of vision and image understanding.

A helpful example of a color-vision application involves the placement verification of colored fuses in auto junction boxes (see Fig. 1). Fuse-block subassemblies are typically populated by a third-tier vendor, then installed into second-tier subassemblies, and, lastly, wired into an automobile chassis.

FIGURE 1. In a typical automotive application, color-coded fuses are pre-inserted into fuse blocks. These blocks are then placed into larger dashboard or firewall assemblies. Fuse ratings and locations vary depending on the make and model of the automobile, and fuses are manufactured in a variety of size, shape, and color specifications.

A misplaced fuse costing a few cents can cause hundreds of dollars worth of rework and overhead when it is discovered in final auto assembly. Even worse, if the defect makes its way into the field, the customer's perception of overall product quality is markedly diminished, and the possible damage, safety, and/or liability issues associated with auto failure due to improper fusing are immense.

Proper position

The key requirement of fuse-block inspection is to verify that all fuses have been inserted in their correct positions. Initially, fuse recognition on the basis of color appears simple. However, examination of the fuses and their nonuniform colors, shadows, and glints reveals that a nontrivial solution is needed. Further complications emerge when fuses with dissimilar ratings possess similar colors. For example, the close color similarity between yellowish 20-A and brownish 5-A fuses presents severe inspection difficulties.

Furthermore, other artifacts are collocated within the color field of each fuse. They include the ends of the embedded metal contacts and imprinted numerals, which generally have different colors from that of the identifying fuse color. Field experience has demonstrated that these artifacts can cause complicated recognition problems for color systems that rely on traditional thresholding and distribution matching extrapolated from gray-scale imaging experience.

FIGURE 2. Popular white-light sources for color-based machine-vision systems include tungsten halogen, metal halide, and fluorescent. The relative outputs of these sources produce tremendous variations in spectral properties and stabilities.

Successful implementation of a robust color-based machine-vision system demands the careful selection of the incoming illumination and optics, inspection algorithms, and the digital color camera. Field practice discloses that a single-chip camera is sufficient to meet the demands of color-inspection applications. For the fuse-block application, the JVC TK-1070U (www.jvc.com) camera is used because of its channel-to-channel stability.

Color techniques

Color is derived from a complex combination of incoming illumination, material interaction, and detection parameters. Because most modern automotive fuses are color-coded to facilitate human recognition, color-based machine vision provides an obvious method to automate correct fuse-placement verification. The techniques of color-based fuse-assembly inspection can be readily applied to a variety of other color-based industrial applications.

The measurement or determination of color can be made in various ways and results in a diverse set of parameters. The effectiveness of color-based recognition techniques varies greatly, depending on the application and the expected results. A successful method in solving fuse-block inspections and similar color-matching applications has been accomplished through joint efforts using WAY-2C (www.way2c. com) color-inspection systems. These systems are based on a patented theoretical information classification method that has proven well suited to this fuse-block inspection application.

Traditionally, variance minimization, correlation-function, and threshold-based algorithms have been used for object recognition in gray-scale images. When used in a color-based system, however, these techniques can prove unpredictable as a result of the violation of key assumptions underlying these approaches. The primary assumption is that the mean value of a distribution approaches the most likely value, a condition that is not met in a majority of multicolored scenes.1

The violation of these assumptions can severely hamper a system's ability to properly execute the required inspection task. Other common factors that can influence the accuracy of color-based machine vision include variations in component colors as supplied by the vendor(s), components with similar color (especially under certain lighting conditions), and the potential signal drift associated with many digital cameras.

Illumination demands

For many machine-vision applications, lighting is the most challenging part of the system design and becomes a major factor when it comes to implementing color inspection. The uniformity and stability of the incoming lighting are usually common causes of unsatisfactory and unreliable performance of gray-scale machine-vision systems.

In color applications, these properties become even more important. If the inspection system is trained to a specific color and the color drifts due to the changing spectral properties of the incoming illumination, the vision system will make incorrect determinations based on the change in color due to lighting, rather than on actual changes in the device under test. In general, when more spatially uniform, temporally stable, and spectrally pure (and stable) lighting is applied in the presence of external factors such as line-voltage fluctuations, temperature changes, bulb aging, and ambient lighting changes, then more reliable results are obtained from the vision system.

FIGURE 3. The standard tungsten-halogen spectral output (red) exhibits more red than blue photons. This output can be balanced with a specialty filter (green) but at the expense of intensity. The blue curve shows the new lamp technology with improved red/blue balance.

In addition to spectral properties, the lighting system must also illuminate the object so that all parameters of interest are distinguishable based on color distribution alone. These parameters include geometrical (lighting structure) effects and interference effects from external factors (ambient and secondary reflections). Experience shows that the best results for blade-type fuses are obtained when light is incident from an angle of about 45° with respect to the normal. This setup minimizes the background effects caused by the color mixing of stray and reflected light from other parts of the fuse block, and, at the same time, it reduces glare from the front face of the fuse. This approach also allows the light to enter into the sides of the fuse and then diffusely emerge from the top, which causes the fuse to "glow" at the corresponding color.

FIGURE 4. Closely matched blue and green fuses are shown under standard tungsten lighting. For proper determination and location of 30-A-green and 60-A-blue fuses under this lighting, a human inspector might find these fuses difficult to distinguish in an image. However, a color-based machine-vision system would easily distinguish between both fuses.

An appropriate incident-light spectral distribution is particularly important for all color-based inspections. The spectrum must be broad and balanced enough to show up any significant color differences within the device under test. A variety of white-light sources are used for these applications, such as fluorescent, tungsten halogen, and metal halide. Each of these sources has advantages and disadvantages, however, the spectral outputs of these dissimilar sources are drastically different (see Fig. 2). Fluorescent and metal-halide lamps are both plasma-type discharge types and are more difficult to regulate than are the hot-filament-type tungsten-halogen lamps. Their associated spectra contain a combination of spectral peaks of high intensity, with backgrounds regions that have little or no output in several color bands. These missing bands can create false color representations, especially if the lamp type is changed.

The stability of the individual spectral curves of the three lamp technologies is also different. These differences can impact the long-term reliability of color-based machine-vision systems. Plasma-type lamps are less stable in spectral output, especially metal-halide lamps, which use a complex chemical composition that changes over time. When properly controlled, the tungsten-halogen lamp is the technology of choice for color-based machine-vision systems. Using sensitive, light-feedback techniques that perform real-time sampling and control of the outgoing spectral radiation ensures a constant intensity and spectral output throughout the life of the lamp.

FIGURE 5. To improve the image quality for human viewers of the color fuse blocks, the lighting solution is to use a 4200 K lamp with additional color-shifting filters to move the color temperature towards 5500 K for a much bluer light. Note that the change in lighting improves the human ability to distinguish the blue and green fuses in the image.

When operated at full intensity, the output from a standard tungsten-halogen lamp has a color temperature of about 3200 K (see Fig. 3). This lamp's output is smooth and continuous with no spikes or dips in the spectra, and its 3200 K color temperature provides a high output at the red end and a low output at the blue end.

Although the light from this lamp appears white to humans, a color CCD camera produces a rich red image. This color variation is due to the imbalance of the lamp's spectral output, and it is further exaggerated by the wavelength-dependent sensitivity of a standard silicon CCD sensor, which has stronger sensitivity to red photons than to blue photons. In many color applications, including the majority of fuse-block applications, the use of a balanced white-light source is preferred in combination with an off-the-shelf single-chip color camera offering red-green-blue output and long-term stability. These camera attributes provide an optimum balance between color quality and cost.

The green curve in Fig. 3 shows the standard method for balancing the output of a 3200 K lamp. A color-shifting or balancing filter can be used to flatten the output so that it has equal intensity over the entire visible range. This filter does provide equal intensities at all wavelengths, but with a reduction in intensity of nearly 60% of the total power. The 4200 K lamp, represented by the blue curve, modifies the standard spectral output of the tungsten-halogen lamp without the use of balancing filters. While this output is not as flat as the daylight corrected spectra, it provides a much whiter light with about a 10% reduction in overall intensity. This 4200 K lamp has become the lamp of choice for most color applications.

White light—not

In many automobile-assembly and similar color-inspection applications, it is not necessary to reproduce the "true" colors of the fuses, only to separate them by their appropriate group and to verify that they are in the right locations. Consequently, lighting designers have some freedom to manipulate the light source to emphasize particular portions of the spectrum that might provide the proper diagnostic information and to de-emphasize or eliminate those wavelengths that contribute less desirable information. This approach is useful in applications where two different fuse values have similar colors. In this case, vision-system designers must be careful to recognize that colors appearing almost identical to the human eye may not appear similar to a camera/algorithm-based system, and vice-versa.

An example of this phenomenon occurs in the case of another class of fuse blocks (see Fig. 4). This application calls for the proper determination and location of 30-A green and 60-A blue fuses. Under normal lighting, a human inspector might find these fuses difficult to distinguish in an image; a machine-vision system would easily distinguish between both fuses.

To improve image quality for human viewers, the lighting solution is to use a 4200 K lamp with additional color-shifting filters to move the color temperature toward 5500 K for a much bluer light (see Fig. 5). Note that the change in lighting improves the human ability to distinguish the fuses in the image. This technique, however, does not make the machine-vision system more robust in classifying the 30- and 60-A fuses. In fact, the machine-vision system finds it more difficult to differentiate the 20- and 40-A fuses. To the human eye, this seems to make little sense.

FIGURE 6. Spectral transmission curves for each fuse were measured. The data quantify the actual photonic distributions that the CCD camera sees, assuming the use of a perfect white- light source with uniform distribution from all visible wavelengths. Comparing spectra for the blue and green fuses explains why the machine-vision system can discriminate between these two colored fuses, even at low saturation.

To clarify this phenomenon, quantitative measurements of the spectral properties of the various fuses were made. The spectral transmission curves for each fuse were measured (see Fig. 6). The data quantify the actual photonic distributions that the CCD camera sees, assuming the use of a perfect white light source with uniform distribution from all visible wavelengths. Comparing the spectra for the blue 60-A fuse with the green 30-A fuse explains why the machine-vision system can readily discriminate between these two colored fuses, even at low saturation.

The measurement data also prove why the vision system has more difficulty in separating the red 40-A and brown 20-A fuses. Here, the spectral curves for these two fuses are similar, yet humans can readily distinguish them. From the data, adding blue light to these two fuses has no effect. In fact, a more reddish light would appear to enhance the characterization, which is exactly the opposite of the first visual inspection. Therefore, the proper measurement of spectral information must be used to make correct decisions regarding input illumination

REFERENCE

1. 1. R. K. McConnell, Vision 15(3), Machine Vision Association of the Society of Manufacturing Engineers, Dearborn, MI (1999).

MIKE MUEHLEMANN is president of Illumination Technologies Inc., East Syracuse, NY 13057; e-mail: muehlemann@illuminationtech. com.