Algorithms breathe life into software

By adding new features to existing software, vendors allow system integrators to tackle various machine-vision applications.

By Andrew Wilson, Editor

Figure 1. MontiVision Imaging Technologies software analyses optical flow in terms of its motion, direction, magnitude, and horizontal and vertical centers of the main motion field.

Up until a decade ago, creating machine-vision and image-processing software to perform a specific industrial-automation task was not easy. Developers could choose to either write their own code or incorporate code from vendor-supplied libraries. Although using vendor-supplied libraries provided a faster time to market, integration of such code into a machine-vision system required an understanding of camera interfaces, frame grabbers, drivers, image formats, and a host of operating systems. Even then, to develop a fully deployable system, specialized graphical user interfaces had to be developed from scratch, resulting in very long development times and high costs.

With today’s machine-vision software, it is possible to develop and deploy fully configured, automated systems at a fraction of the cost and in weeks rather than months. Several factors have led to this progress. First and foremost has been the introduction of tools for the ubiquitous Windows operating system. These include language tools such as compilers and translators, communications, networking and client-server utilities; off-the-shelf (often free) libraries and DLLs; database interfaces; and easy-to-integrate HTML-based graphical user interfaces.

One tool, Microsoft DirectShow, is designed to simplify basic playback, format-conversion, and image-capture tasks while providing access to the underlying stream-control architecture for applications that require custom solutions. This multi-media architecture divides the processing of tasks into steps known as filters that can be connected in different ways. Recently, a number of companies have taken advantage of this architecture in their machine-vision offerings.

MontiVision Imaging Technologies, for example, has incorporated these algorithms as DirectShow filters into its DirectX-based machine-vision development kit. These include a blob counter that evaluates a list of properties for each blob it detects in the processed image and provides functions for preprocessing the input image and a filter mechanism to filter unwanted blobs by size.

According to Marc Cymontkowski, president, the company has also enhanced its 2-D Data Matrix decoder to support PDF417 2-D and common linear barcodes. The company also has added an advanced routine that analyses optical flow in terms of its motion, direction, magnitude, and horizontal and vertical centers of the main motion field (see Fig. 1).

Data Translation and Tordivel AS also have added supports for DirectShow 8.0-compatible FireWire and USB cameras in their latest products. While Data Translation’s Vision Foundry software supports DirectShow 8.0-compatible FireWire or USB cameras with uncompressed data in either 8-bit monochrome or 24-bit RGB format, Tordivel has added USB camera support using DirectX 8.1.

HIGHER LEVELS

In the latest versions of their software packages, vendors also are adding tools to allow image-analysis functions, such as color space transforms, optical-character reading, geometric pattern-matching, and calibration, to be performed more easily. For the systems developer, understanding which functions can solve a particular problem is an easier task than specifying the underlying image-processing routines on which they may be based. This is especially useful in applications that require simple line, blob, color, or orientation measurements to be made. With the introduction of these tools, many software packages now allow system integrators to build industrial automation systems more rapidly.

Figure 2. National Instruments geometric pattern-matching algorithm allows rotational and lighting invariant parts within images to be located independent of scale and occlusion by other parts within the image.

Feature-based optical character recognition (OCR), a new feature added to the Matrox MIL library, for example, has been developed for reading character strings at any scale factor in poor or nonuniform lighting conditions. Useful in applications for which the location of the string and font are unpredictable, such as automobile license-plate reading, the module includes a universal font that can be used in simple applications, bypassing font training altogether. Since the MIL feature-based OCR module supports reading multiple fonts simultaneously, additional fonts and characters also can be used.

EasyMatrixCode, part of the Euresys Mark Inspection libraries, automatically detects and/or verifies 2-D Data Matrix codes. The algorithm used can find and decode matrix codes under extreme lightning conditions (nonuniform, poorly contrasted, or overexposed images), difficult machine setups (small sized, skewed, or nonsquare cell matrix codes), or special applications such as textured, blurred, or noisy images.

LOCATING OBJECTS

Locating specific objects within images is an important task for many industrial-automation systems. Several vendors now offer geometric pattern-matching software that can perform noncontact alignment in machine-vision systems. These include Adept Technology, Cognex, Logical Vision (a subsidiary of Coreco Imaging), Euresys, Matrox, and Mutech, all of which offer PC-based packages (see Vision Systems Design, May 1999, p. 50).

At NIWeek (Aug. 14-17, 2004), National Instruments showed a new geometric pattern-matching algorithm and a series of image-classification tools for its IMAQ Vision software. Bundled in Version 7.1, the geometric pattern-matching tool allows rotation and lighting invariant parts within images to be located independent of scale and occlusion by other parts within the image. After defining a template within the image using a series of editing tools, the software locates specific edges within the image and builds a vector map of the part to be recognized. This is then used to locate matching parts and provide a confidence match of similar parts within the image being inspected (see Fig. 2).

To complement this pattern-matching, the company’s neural-network-based image-classification tools have been developed for fast product sorting applications. Rather than perform repetitive pattern-matching functions, the software is taught specific known patterns and makes a “best guess” based on acquired data. According to product manager Kyle Voosen, Gefra used the software in its Optisort fastener-inspection system that was shown at National Manufacturing Week (March 7-10, 2005; Chicago, IL, USA).

Figure 3. Cognex PatFlex can locate and inspect labels on cylindrical surfaces, such as cans and bottles, and account for surface irregularities that can occur in packages such as pouches and bags.

Also capable of handling arbitrary illumination changes, translation, rotation, occlusion, and scaling, MVTec Halcon’s shape-based pattern-matching algorithm has been extended to model objects that may be composed of components that can move with respect to each other. According to the company, such compound objects could until now only be recognized by creating multiple rigid models, one for each object part. The company claims that, by representing all object parts with one common model, the matching technology is faster, requiring no postprocessing of the result because multiple instances of the compound object are inherently grouped.

ShapeFinder 2, part of the Stemmer Imaging Common Vision Blox (CVB) software, uses a gradient-based approach for the initial object search, employing a lower-resolution image to accelerate the process. The system then searches the high-resolution image at those positions at which candidate objects already have been located. Even at this stage, the results are accurate enough for pure classification tasks. For ultraprecise measurement tasks and extreme pick-and-place requirements, CVB ShapeFinder 2 “Precision Booster Technology” can be activated to subsequently perform a gray-scale correlation analysis at the relevant position.

MACHINE VISION

Realizing the need to offer solutions rather than algorithms or callable routines, some machine-vision software vendors have taken this one step further by building on their own software packages to provide solutions for the semiconductor, telecommunications, and packaging industries. Others have incorporated third-party software routines into their products by licensing such software or purchasing the assets from other third-party vendors. By offering this high-level software, system integrators can leverage or build on an existing solution based on open, user-configurable software development tools. Using these packages can both radically shorten development times and reduce costs.

Because contour changes and topographical variations in surface patterns can occur during production when using flexible packaging materials, sophisticated high-level software routines are required. The newest member of the Cognex PatMax family, PatFlex, enables a vision system to locate an object, feature, or pattern whose perspective has changed or whose surface is curved, warped, wrinkled, or stretched. Greater perspective tolerance provides machine design flexibility, allowing cameras to be mounted at an angle to the work piece. This enables the optical verification of characters marked on curved surfaces, where perspective and bow distortions are common.

PatFlex tolerates skew and aspect compression of the mark and distortions caused by vibrations in the marking operation. In addition, the software can locate and inspect decorative labels on cylindrical surfaces and account for surface irregularities that can occur in packaging operations using flexible shrink-wrap films. PatFlex is also suitable for applications where the object changes shape during production, such as in packaging operations employing nonrigid pouches and bags (see Fig. 3 on p. 45).

One of the first companies to recognize the need for high-level machine-vision software for semiconductor applications, RVSI Acuity CiMatrix offers several products built on its Visionscape machine-vision platform. These include Visionscape BGA Inspection, a system for inspecting bumps on standard BGA devices, wafer-scale devices, flip-chip die, and wafers. For inspection of marked semiconductor packages, Visionscape Mark Inspection can recognize incomplete characters, character voids, incorrect characters, as well as mark orientation.

Rather than develop its own algorithms, DVT recently purchased MTI Machine Vision, a developer of semiconductor-wafer reading software. DVT obtained the rights to the company’s 2-D code for part and package identification, as well as OCR and 2-D decoding software for 300-mm wafer applications. MTI Machine Vision pattern-recognition software achieves subpixel accuracy on textured images under variable lighting conditions to enable robotics equipment to choose alignment patterns with or without operator assistance. DVT expects the first products to include wafer-reading OCR, barcode and 2-D identification algorithms, and alignment and defect-detection software.

Whether such off-the-shelf packages can be used in “real-time” systems is also an important factor. Real-time systems are said to be deterministic because they provide a definitive response time when performing a specific task. Unfortunately, machine-vision algorithms can never be deterministic. If, in a cell-counting application, for example, one image has more cells than another, then an automated system will, by nature, take longer to determine the number of cells in the image with the larger number of cells. Thus, although some vendors may offer real-time operating systems that support their software, it is still up to the system integrator to benchmark the machine-vision system.

In deciding on any type of software, system integrators must also understand the types of cameras, operating systems, industrial I/O, motion-control devices, and networking devices supported. Although some vendors offer very sophisticated software, it may only be supported by their own hardware or hardware from a few other vendors. Business factors such as the cost of the product, reputation of the vendor, any prior experience the developer has with a particular package, and licensing fees that may be required also play an important part in this decision.