PC tools speed machine-vision software deployment

Machine-vision software packages are harnessing the power of PC development tools to enable rapid deployment of imaging systems.

By Andrew Wilson, Editor at Large

In the past, systems integrators were offered C-callable libraries to help them develop machine-vision systems. Although these libraries proved effective, development efforts were often time-consuming and limited by the systems integrators' understanding of the specific toolkit being used.

Currently, thanks to the efforts of Intel Corp. (Santa Clara, CA) and Microsoft Corp. (Redmond, WA), developers of image-processing software toolkits are offering OEMs MMX-enhanced versions that can be configured in easy-to-understand graphical environments. Indeed, many software packages use machine-vision libraries with custom code developed by implementing Visual C++ or Visual BASIC (see table on p. 72). And, while many packages are tied to company-specific hardware, others are hardware-independent, allowing systems integrators to choose from a number of off-the-shelf frame grabbers, image processors, and display controllers.

Click here to enlarge image

FIGURE 1. DS GmbH NeuroCheck software uses Microsoft's Object Linking and Embedding (OLE) automation interface; it allows developers to add functionality without making changes to the main body of the program. After a plug-in DLL has been successfully loaded, functions can be selected and integrated into an image-processing routine.

Says Andre By, president of Automation Engineering Inc. (AEI; Woburn, MA), "The traditional approach of using machine-vision libraries with custom code developed using Visual C/C++, Visual BASIC, or Java has been made more effective in the last few years due to falling prices for available third-party libraries and more-comprehensive analysis tools available in these libraries. The development and debugging environments for these software languages also continue to improve," he adds.

"When building image-processing applications, the main part of the inspection problem can often be solved using built-in standard functions. However, a few requirements may not be able to be met," says Christian Demant, general manager of DS GmbH (Remseck, Germany). "Often these requirements are so specific to the application that the cost and the implementation overhead needed to integrate the solution into a standard software package are prohibitively large," he adds.

Because of such limitations, many image-processing software libraries, including DS GmbH NeuroCheck, use Microsoft Object Linking and Embedding (OLE) automation interface that allows developers to add functionality without making changes to the main body of the program. In NeuroCheck, for example, developers can load a plug-in dynamic link library (DLL) and integrate it into an image-processing routine (see Fig. 1).

Click here to enlarge image

FIGURE 2. The Matrox Imaging Library (MIL) is bundled with ActiveMIL, a collection of ActiveX controls for managing image-capture, transfer, processing, analysis, and display functions. Integrated into Microsoft Visual BASIC or C++, ActiveMIL permits developers to build imaging applications using both C-programming (left) or ActiveX (right) controls.

As a technology for exchanging objects between applications, OLE allows data and programmable objects to be used by other programs. These programmable objects expose certain interfaces to the system, permitting other OLE-capable applications to control the original programmable object. Controlling another application's programmable objects is called OLE or ActiveX automation.

Click here to enlarge image

The technology upon which OLE is built—the Component Object Model (COM)—specifies a set of rules for the creation of binary objects that can communicate with each other. By following these rules, programmable objects can be written in an OLE-capable programming language and accessed by an OLE-capable application. Recently, COM has been extended to function between various computers connected over a network, which is called distributed COM.

Because the COM specification is independent of the programming language, an OLE client can be written in an OLE-capable programming language such as Visual BASIC, Visual C++, or Borland Delphi. "Here," says Demant, "the decisive factor for the speed of execution is dependent on the number of calls to the OLE interface through the Windows Automation Manager."

The most recent extension of COM/OLE is called ActiveX; it enables software components to interact with one another in a networked environment, regardless of the language in which the components were created. An ActiveX control is a user-interface element created using ActiveX technology. "ActiveX controls are small, fast, and powerful, and they make it easy to integrate and reuse software components," says Demant.

In many PC-based image-processing programs, ActiveX controls are used to manage image-processing functions. The Matrox Imaging Library (MIL) from Matrox Electronic Systems Ltd. (Dorval, Quebec, Canada), for example, is bundled with ActiveMIL, a collection of ActiveX controls for managing image capture, transfer, processing, analysis, and display functions. Integrated into Microsoft Visual BASIC or C++, ActiveMIL allows developers to build imaging applications using both C programming or ActiveX controls (see Fig. 2 on p. 67).

MMX technology

While most machine-vision software vendors have embraced Microsoft's development tools, they have also used Intel's multimedia extensions (MMX) to speed traditional image-processing functions such as convolution and morphology. These extensions add new data types that allow handling of 64-bit data by reassigning 64 bits of each of the eight 80-bit floating-point registers as MMX registers.

Click here to enlarge image

FIGURE 3. The Imaging Technology Sherlock32 Windows-based machine-vision software is intended to simplify development and deployment of machine-vision applications. Once the application is developed, the company's MVTools C/C++ library or DLLs allow OEMs to write their own Visual BASIC or Visual C++ front ends to access MVTools machine- vision algorithms.

However, because the MMX hardware and floating-point unit share registers, floating-point and MMX instructions cannot normally be intermixed without severe performance penalties. It takes about 50 clock cycles to toggle the floating-point register set between floating-point use and MMX operation.

Despite this impediment, many embedded image-processing applications do not require switching between MMX and floating-point performance. And manufacturers such as Imaging Technology Inc. (Bedford, MA) and National Instruments (Austin, TX) have been fast to adopt the MMX instruction set into their image-processing software. The Imaq Vision 4.1.1 software from National Instruments, for example, incorporates a low-level MMX library.

"As a library of machine-vision and image-processing functions for LabView, BridgeView, ActiveX containers, and LabWindows/CVI, many of the Imaq vision functions have shown significant performance gains for filtering, thresholding, arithmetic, logical, and morphological operations," says John Hanks, vision marketing manager at National Instruments. "However, it is important to remember that the performance gains for MMX image processing depend on the algorithm or function. For example, the Imaq histogram function does not show significant performance gains because it is a sorting routine that cannot easily take advantage of the MMX architecture. However, functions such as add, threshold, and multiply show increased performance," he says.

Increased flexibility

"While approaches that use ActiveX automation provide the most flexibility in the features that can be included in machine-vision applications, custom software must still be developed for each application," says AEI's By. "This is not viable for many systems integrators and equipment manufacturers. Equipment manufacturers that use machine vision for flexible automation equipment need a process-oriented "programming" approach for their customers that can be learned quickly by nonprogrammers," he adds. For these control applications, developers need to teach machine control sequences that incorporate machine vision, input/output, and HMI functions.

Accordingly, companies such as AEI and Imaging Technology have provided equipment manufacturers, systems integrators, and end users with point-and-click teaching environments that define the machine-vision system functions. These environments permit configurable machine-vision software to be set up quickly and easily and allow different functions to be executed with little training.

Click here to enlarge image

FIGURE 4. The CommotionTechnology ControlFoundry 2.0 PC-WorkCell control application helps developers design motion-control, vision-processing, and analog and digital input/output functions, including DeviceNet, 3-D simulation, and graphical programming from a PC. To provide vision-processing capability to the package, ControlFoundry incorporates the MatroxMIL 6.0 image-processing library from Matrox.

Imaging Technology's Sherlock32, for example, is a Windows-based machine-vision software environment intended to simplify development and deployment of alignment, gauging, inspection, assembly verification, and machine-guidance tasks (see Fig. 3 on p. 69). With no programming needed, system integrators and end users can reduce development time and speed the time to market of their application.

Once the application is developed, Imaging Technology MVTools C/C++ library or DLLs allow OEMs to write their own Visual BASIC or Visual C++ front ends to access MVTools machine-vision algorithms. Alternatively, OEMs who prefer programming vision tasks via OCX can run Sherlock32 in server mode and communicate with the vision application using a Visual BASIC or a Visual C++ front end.

DT Vision Foundry software from Data Translation (Marlboro, MA) also can be used at different levels. Developers using this package can set up stand-alone machine-vision applications using predefined image-processing tools. Alternatively, DT Vision Foundry can be used as a base for the addition of user-written custom tools that include a custom graphics user interface (GUI) or as an object-oriented application programming interface (API) for custom machine-vision applications. Since most imaging applications differ, developers can use several methods, including a Basic script tool, point-and-click script tool, or Visual C++, to create their applications.

Integrating functions

Advanced vision functions, motion control, and customizable user interfaces or teaching environments are common needs of automation-equipment manufacturers and systems integrators. Realizing this, companies such as AEI and Commotion Technology (San Francisco, CA) include open-architecture motion-control libraries and configurable application software for machine vision as part of their software offerings. These packages can be extended with option modules and hybrid development packages that support point-and-click programming, scripting, and motion control.

Click here to enlarge image

The AEinspect 2.3 software from AEI allows nonprogrammers to develop and prototype basic machine-vision applications using different PC-based frame grabbers. To meet machine control requirements, AEI developed the FlexAuto package for Windows NT. FlexAuto 4.0 allows developers to implement PC-based machine control integrating motion control, machine vision, I/O, and a customizable end-user teaching environment. FlexAuto 4.0 also supports different frame grabbers, motion-control cards, and I/O cards to allow automation-equipment manufacturers and systems integrators to select the most appropriate mix of hardware to implement a machine-control application.

Commotion Technology (San Francisco, CA) is also developing PC-based systems that provide vision systems designers with motion-control functionality. The company`s ControlFoundry 2.0 PC-WorkCell control application, for example, allows developers to design motion control, vision processing, and analog and digital input/output, including DeviceNet, 3-D simulation, and graphical programming from a PC (see Vision Systems Design, July 1999, p. 30). To provide vision-processing capability to the package, Commotion Technology's ControlFoundry incorporates the MatroxMIL 6.0 image-processing library from Matrox (see Fig. 4).

In choosing an image-processing package, developers must also carefully consider their final application. While most standards-based application packages run on PC-based systems and incorporate ActiveX technology and MMX extensions, many have been developed to support company-specific hardware from the software developer. Choosing such packages will ultimately tie systems integrators to that company's hardware. Software such as Common Vision Blox from Integral Vision (Farmington Hills, MI) and Halcon from MVTec Software (Munich, Germany) support hardware from a number of different PC-base frame-grabber and image-processing board vendors. Developers must therefore carefully consider whether such supported hardware will meet their demands.

Pattern-matching software eases machine vision

Since Vision Systems Design last covered pattern-matching software (May 1999, p. 50), several companies have introduced or upgraded their software packages. After Cognex (Natick, MA) first introduced this concept in its PatMax product, many other vendors either incorporated pattern matching as ActiveX (OCX) controls in their imaging toolkits or development systems or as C or C++ callable routines.

MIL 6.0 pattern-matching software from Matrox (Dorval, Quebec, Canada), for example, makes image-processing and analysis functions available as ActiveX (OCX) controls with the ActiveMIL package. Using a search-and-locate strategy to find predefined models within target images, the software can find target models rotated by angles up to 360¦ with a translational accuracy of up to 1/40 pixel and a rotational accuracy of up to 1/10°.

Like Matrox, Imaging Technology (Bedford, MA) also offers a search tool, called Smart Search, in its MVTools, a C/C++ run-time library of vision software tools that include NGC, blob, caliper, edge, line, histogram, morphology, and image- processing functions. Capable of locating a pattern to within a pixel, the software uses a neural-network-based approach to compute the match to subpixel accuracy.

Operating in development environments that use standard ActiveX controls, Visual Basic and/or Visual C++, HexSight from HexaVision (Sainte Foy, Quebec, Canada) also uses the geometry of parts to locate them in a two-dimensional field of view. Algorithms used in the Active X-based software-development system find the contours of a part in the image and identify the part's position and appearance. These algorithms allow the location and inspection of randomly oriented parts with rotational variances from 0° to 360° to a precision of 1/100° and 1/40 pixel.

Align Version 2.0 for Windows NT software from MuTech (Billerica, MA) can also be supplied with additional DLLs and OCXs to support the implementation of the program in Visual Basic. In operation, the software's Locate function uses a NGC algorithm to perform the comparison between a model and a target on a pixel-by-pixel basis.

PHOTO 6 Last year, Coreco (St-Laurent, Quebec, Canada) also entered the pattern-matching software market with its fastAlign, part of the company's fastSeries of MMX optimized "C" callable vision libraries. As a member of the FastSeries C libraries, the FastAlign library consists of high-level functions to perform both single and multiple pattern recognition at rates up to 200 patterns/s. Also included is support for computing subpixel accurate results at up to 1/64 pixel and handling of pattern variations caused by rotation and scale.

Last month, Datacube (Danvers, MA) also announced MMX optimized pattern-matching software—vsFind. Designed to run under Windows 95/98/NT, the software uses mathematical-correlation methods to find multiple instances of the same template in one image. Alternatively, different templates can be found in the same image. Demonstration software, downloadable from the company's Web site (www.datacube.com) can be used to benchmark the software on PC systems.