Smart cameras challenge host-based systems in industrial applications
Incorporating an image sensor, FPGA, CPU and I/O capability into a single unit, smart cameras offer developers a viable alternative to host-based vision systems.
In industrial process monitoring where space constraints are limited, the use of numerous cameras interfaced to PC-based machine vision systems may not be viable. In such cases, the use of smart cameras that incorporate the features of both cameras and PCs provide systems integrators with the option of localized pass/fail decision making, I/O part rejection and networked management capability while reducing the footprint of their systems.
Over the past ten years, the demand for smart cameras has steadily grown in machine vision applications. However, as pointed out by Yu Shi of National Information and Communications Technology Australia (NICTA; Eveleigh, N.S.W., Australia; www.arc.gov.au/national-ict-australia) in "Smart Cameras: A Review," these cameras will increasingly be deployed in healthcare, entertainment and education (http://bit.ly/VSD-YUSHI).
Each of these applications demand a different type of smart camera. While security cameras produced in high-volume may demand low-cost imagers, processors, memory and wireless interfaces, those developed for industrial machine vision applications require additional capabilities. For those wishing to deploy smart cameras for such applications, there exist an array of products that incorporate different type of image sensors, CPUs, DSPs, FPGAs, I/O, software and internal and external lighting capabilities.
Multiple-processor architectures are useful in providing the image processing throughput required in machine vision applications since vision tasks can be partitioned between different processors. For example, while point and similar processing tasks such as histogram equalization and image filtering are most effectively performed by FPGAs, the task of statistical pattern recognition may best be implemented on general purpose CPUs. By segmenting these tasks among multiple processors, image processing tasks can be efficiently performed, thus reducing latency and decreasing processing times.
While every smart camera vendor has adopted their own particular computer architecture, the differences between them can have a significant impact on system performance. As Abdul Waheed Malik of the Mid Sweden University (Sundsvall, Sweden https://www.miun.se), indicates in his paper "Comparison of Three Smart Camera Architectures for Real-Time Machine Vision System," while commercially-available, software-based smart cameras may exhibit a higher latency and consume more power than other types of designs, they benefit from easier software programming (http://bit.ly/VSD-3SC).
In such designs, an FPGA may perform low-level image processing functions such as defective pixel correction, gamma correction or Bayer interpolation before an image is processed by a CPU, DSP or GPU. By off-loading these tasks to the FPGA, the processor in the camera is free to perform higher-level image processing tasks.
Just as the smart camera's architecture is important, the ease of using and/or programming such cameras to perform a specific task at the speed required is key for systems integrator. While smart cameras are available in a number of different configurations with various types of image sensors, I/O and lighting capabilities, vendors may supply products to perform specific imaging tasks such as barcode reading. These are generally tailored to use the camera's on-board FPGA and CPU and cannot be configured to perform other tasks.
Many systems integrators, however, require smart cameras that are capable of performing multiple image processing operations such as image enhancement, feature location, object measurement, presence/absence detection or other machine vision tasks. Here, the developer is faced with a number of options. To meet these needs, many smart camera vendors offer products with their own software and graphical user interface (GUI) development systems. While the use of such cameras may be limited to the company's own software, the systems developer is guaranteed that the vendor has an intimate knowledge of both the smart camera and the vision software it employs.
Some vendors - most notably those with an established presence in the development of machine vision software - have recognized the emergence of third-party smart cameras similar in design to the products they offer. Rather than ignore this opportunity, many of these now allow other smart camera vendors to incorporate their software into their products. While third-party camera vendors may offer such capability, however, the support they provide may be more limited than vendors of both smart camera systems and machine vision software. By allowing such developments to occur, sophisticated systems integrators can use off-the-shelf software packages and third-party cameras to develop the most efficient and effective solution for their machine vision tasks.
Barcode reading
Today, a number of companies offer smart cameras - often referred to as smart image sensors-specifically tailored to barcode reading. These include the DataMan and MX products from Cognex (Natick, MA, USA; www.cognex.com), the Matrix series from Datalogic (Bologna, Italy; www.datalogic.com) , the SR Series from Keyence (Osaka, Japan; www.keyence.com), MicroHAWK readers from Microscan (Renton, WA, USA; www.microscan.com), the Lector series from SICK (Waldkirch, Germany; www.sick.com) and the BOA Spot ID from Teledyne DALSA (Waterloo, ON, Canada; www.teledynedalsa.com)
While each of these companies has employed imagers and processors specifically to target barcode reading, many have realized that merely reading or verifying the integrity of such barcode data must be supplemented with real-time monitoring systems and software. By implementing such networked systems, plant managers can identify and classify any reading or verification errors and visualize data about their causes, such as a faulty barcode printer and thus increase the percentage of codes that are correctly read, improve the efficiency of the machines on the production line and reduce waste.
Several examples of such real-time monitoring systems exist including the Package analytics software from Sick, the WebSentinel PLUS from Keyence and the Cognex Explorer Real Time Monitoring (RTM) system (Figure 1). By interpreting this image data, rather than just reading or verifying it, such systems can empower those in management with invaluable data about the performance of their smart-camera-based barcode readers and the status of other peripherals on their industrial network.
Proprietary solutions
Although many smart camera designs are similar, manufacturers recognize that the value-added to such products is attained by the machine vision tasks the camera can perform. Keeping such software tailored to a specific device provides many vendors with a means to more closely control their product offerings and the tasks their customers may need to perform. Most notably, both Keyence and Omron (Kyoto, Japan; www.omron.com) offer smart cameras with their own proprietary software to perform specific imaging tasks.
While Keyence's Vision Sensor IV series can perform such functions as edge presence, contour and color difference measurements, for example, the FQ2 Series from Omron with its Shape Search III tool can search and match registered patterns. This is especially useful in identifying labels of a specific color and size and, at the same time, measuring the area or center position of the label.
Proprietary hardware/software solutions are not just offered by Japanese vendors. Datalogic, for example, supports its P-Series, A-Series and T-Series of smart cameras, with the company's Impact software to provide developers with over one hundred inspection tools and user interface controls to create machine vision inspection programs (Figure 2). To support its VE Series of smart cameras, Banner Engineering (Minneapolis, MN, USA; www.bannerengineering.com) offers its Vision Manager software for tasks such as item detection, part positioning, feature measurement and flaw analysis. For its part, the Neo from Soliton Technologies (Bangalore, India; www.solitontech.com), a DSP-based smart camera, is supplied with the company's own Soliton Vision Library (SVL). OEMs wishing to tailor the camera can, however, use Code Composer Studio from Texas Instruments (Dallas, TX, USA; www.ti.com) to develop their own algorithms.
Software is key
Just as companies such as Keyence and Omron offer proprietary software that can only be configured with their cameras, established software suppliers also offer their own smart cameras. Among these, Cognex, Teledyne DALSA, Matrox Imaging, Microscan and National Instruments (Dallas, TX, USA; www.ni.com) all offer products that can be used with their machine vision software.
With long histories of developing machine vision software, these companies have recognized the need of systems developers who require an easy way to program their smart cameras. To do so, they have developed simplified user interfaces that leverage image processing algorithms that may have been, in the past, offered as C-callable functions. By doing so, systems developers need not perform complex software development tasks, freeing them to concentrate on the inspection task at hand.
Such development tools are offered in a number of different configurations to allow integrators to configure smart cameras using easy to understand interfaces. Several different options exist to program smart cameras using these interfaces. For its In-Sight series of smart cameras, for example, Cognex's EasyBuilder software allows functions such as pattern matching, filtering, defect analysis and OCR tasks to be accomplished by selecting a series of functions from a menu.
Similarly, Teledyne DALSA's iNspect Express interface guides developers to set up vision tasks to perform functions such as presence/absence inspection, part alignment, measurement and surface inspection (Figure 3). Running on its BOA series of smart cameras, the software also includes a scripting function allowing communication with 3rd party devices over Ethernet or RS-232 serial interfaces. In the development of its MicroHAWK smart camera series, Microscan offers its AutoVISION GUI for tasks such as barcode reading, part location, counting, presence/absence detection - software that is scalable with the company's VisionScape software that can be used for more advanced applications.
Taking a different approach, Matrox offers a flow-chart method with which to program their Iris GTR smart cameras. Built using the company's Matrox Imaging Library (MIL), the Matrox Design Assistant is an integrated development environment (IDE) that allows developers to build applications without the need for conventional programming. The IDE also enables graphical operator interfaces to be developed for specific applications.
Third-party solutions
While many proprietary smart cameras are dedicated to tasks such as barcode reading, programmable smart cameras allow a number of machine vision tasks to be performed. Rather than choose a smart camera from a vendor that offers both smart cameras and software, however, integrators can opt for third-party cameras. Because many of these cameras are based on Intel or AMD processors, developers can leverage the software developed by both companies that offer smart cameras and software and suppliers of PC-based machine vision software.
With its Intel Atom processor, for example, the NEON-1021 from ADLINK Technology (New Taipei, Taiwan; www.adlinktech.com), supports a number of software packages including MERLIC and HALCON from MVTec (Munich, Germany; www.mvtec.com), Common Vision Blox from Stemmer Imaging (Puchheim Germany; www.www.stemmer-imaging.de), VisionPro from Cognex, MIL from Matrox (Dorval, QC, Canada; www.matrox.com), and Sherlock from Teledyne DALSA.
Third-party machine vision software packages such as the Graphical Adaptive Vision Studio 4.8 from Adaptive Vision - a Division of Future Processing (Gliwice, Poland; www.adaptive-vision.com) are also supported by a number of smart camera vendors. The company's software, which uses a data-flow based software interface, can be programmed for such tasks as camera calibration, barcode reading, one and two-dimensional measurements and template matching (Figure 4).
Programs created in Adaptive Vision Studio can be run on NEON smart cameras from ADLINK, the Corsight from New Electronic Technology (NET; Finning, Germany; www.net-gmbh.com), the Woodpecker1 from Roseek (Shanghai, China; www.roseek.com) and the xPlatform from XIMEA (Münster, Germany; www.ximea.com). At present, XIMEA's version with Intel's NUC maintains support for four Libraries - LabVIEW from National Instruments, MATLAB from MathWorks (Natick, MA, USA; mathworks.com), HALCON from MVTec and the Open Source Computer Vision Library (OpenCV; www.opencv.org) while providing limited support for a number of other software packages.
Systems developers wishing to deploy smart cameras in their production facilities have a number of different options: choosing dedicated fixed function devices, programmable cameras from established hardware and software vendors, or opting for third-party PC-based cameras. Perhaps more importantly than the price/performance each of these offer is the functionality and level of software support that each manufacturer offers.
Companies mentioned
ADLINK Technology
New Taipei, Taiwan
www.adlinktech.com
Adaptive Vision - a Division of Future Processing
Gliwice, Poland
www.adaptive-vision.com
Banner Engineering
Minneapolis, MN, USA
www.bannerengineering.com
Cognex
Natick, MA, USA
www.cognex.com
Datalogic
Bologna, Italy
www.datalogic.com
Keyence
Osaka, Japan
www.keyence.com
MathWorks
Natick, MA,USA
www.mathworks.com
Matrox
Dorval, QC, Canada
www.matrox.com
Microscan
Renton, WA, USA
www.microscan.com
Mid Sweden University
Sundsvall, Sweden
https://www.miun.se
MVTec
Munich, Germany
www.mvtec.com
National Information and Communications Technology Australia (NICTA)
Eveleigh, N.S.W., Australia
www.arc.gov.au/national- ict-australia
National Instruments
Austin, TX, USA
www.ni.com
New Electronic Technology (NET)
Finning, Germany
www.net-gmbh.com
Omron
Kyoto, Japan
www.omron.com
Open Source Computer Vision Library (OpenCV)
www.opencv.org
Roseek
Shanghai, China
www.roseek.com
SICK
Waldkirch, Germany
www.sick.com
Soliton Technologies
Bangalore, India
www.solitontech.com
Stemmer Imaging
Puchheim Germany
www.stemmer-imaging.de
Teledyne DALSA
Waterloo, ON, Canada
www.teledynedalsa.com
Texas Instruments
Dallas, TX, USA
www.ti.com
Ximea
Münster, Germany
www.ximea.com
About the Author
Andy Wilson
Founding Editor
Founding editor of Vision Systems Design. Industry authority and author of thousands of technical articles on image processing, machine vision, and computer science.
B.Sc., Warwick University
Tel: 603-891-9115
Fax: 603-891-9297