Smart cameras evolve to meet expanding machine vision needs

Smart cameras—typically those combining lighting, image sensors, software, and I/O—have long eased the task of machine vision system design. Capabilities and specifications of smart cameras have grown, including models with larger image sensors, emerging embedded vision cameras that function as smart cameras, and new cameras capable of performing deep learning and artificial intelligence tasks.

Larger sensors

Historically, smart cameras available in the machine vision market feature image sensors in the range of VGA to 2 MPixels. Growing market demands for inspecting larger parts and identifying smaller defects in images have led to a paradigm shift as the resolution of smart camera image sensors increases.

“Over the last 12-18 months, image sensor resolution in smart cameras has increased,” says David Dechow, Principal Vision Systems Architect, Integro Technologies (Salisbury, NC, USA; www.integro-tech.com). “Right now, 5 MPixel falls into the category of higher-resolution, but very soon, it will become the standard resolution, as 9, 10, 12, 16, and 20-plus MPixel image sensors become the new high-resolution, even in smart cameras.”

Several companies today offer smart cameras with image sensors larger than 5 MPixels, including Cognex (Natick, MA, USA; www.cognex.com), which offers an entire range of image sensor options.

While In-Sight 2000 vision sensors feature image resolution up to 1.2 MPixels, the company also offers several models with larger CMOS imagers, including the In-Sight 7000 series with monochrome and color imagers ranging from 0.3 MPixels (VGA) up to 5 MPixels. In-Sight 8000 smart cameras also feature imagers up to 5 MPixels, while the In-Sight 9912 system (Figure 1) features a 12 MPixel imager. For line scan imaging, the In-Sight 9902L line scan system acquires up to 16,000 lines of 2,000 pixels per line to produce a 32 MPixel image that, according to the company, can detect even the smallest features and defects.

“Increasingly more machine vision applications require the detection of smaller features in the image, or imaging very large parts, or both at the same time,” says Robb Robles, Senior Manager, Product Marketing, In-Sight Products. “Think large automotive parts, for example engine blocks and door panels.”

“Traditionally,” he continues,” these applications required multiple cameras, but now, a single smart camera addresses applications that require detection of small features in a large field-of-view.”

Having acquired Imaging Solutions Group (Fairpoint, NY, USA; www.isgcameras.com) in September 2018, JADAK (North Syracuse, NY, USA; www.jadaktech.com) now offers a range of smart cameras in its Allegro series. These cameras feature numerous Sony sensors, including the 3.2 MPixel IMX252 and IMX265, the 5.1 MPixel IMX264, 8.9 MPixel IMX255 and IMX267, and the 12 MPixel IMX304. Additionally, the cameras use the CMV2000, CMV4000, and CMV12000 CMOS image sensors from ams (Premstaetten, Austria; www.ams.com), and offer numerous interface and board level options, including GigE Vision, direct interface to NXP i.MX6 camera port, MIPI out, and customer specific interfaces. Allegro smart cameras target disparate applications, including machine vision; medical, science, and microscopy; security, surveillance, and traffic; and environmental monitoring applications.

Apart from the Allegro series, JADAK also has the 5 MPixel SC-6V, the 5 MPixel SC-6L camera with high-intensity illumination, the 640 x 480 VIBE camera, the 0.5 MPixel FM-5V camera, and the 0.5 MPixel FM-6V camera. These fixed-mount cameras suit a range of applications including machine vision, barcode reading, OEM applications, lottery, medical devices, robotics, and kiosks.

In its Iris GTR smart cameras—available in standard models and with integrated Matrox Design Assistant X Software—Matrox Imaging (Dorval, QC, Canada; www.matrox.com) offers cameras with various image sensor options and an Intel Celeron N2807 embedded processor. Available in either monochrome or color formats, these smart cameras target various machine vision inspection applications and have GigE connectivity and PYTHON CMOS image sensors from ON Semiconductor (Phoenix, AZ, USA; www.onsemi.com), including the 0.3 MPixel PYTHON 300, 1.3 MPixel PYTHON 1300, 2.3 MPixel PYTHON 2000, and the 5.3 MPixel PYTHON 5000 sensors.

Manufacturing numerous smart cameras with sensors beyond 5 MPixels, Eye Vision Technology (Karlsruhe, Germany; www.evt-web.com) offers the 2000, 3000, 4000, 5000, and ZM series of EyeCheck smart cameras. In the 5000 series, the company also offers models with 6.2 (EC5700) and 19.9 MPixel (EC5900C) CMOS image sensors. Made in color and monochrome versions, the smart cameras feature various processing options, including ARM Cortex-A8 processors, ARM dual-core Cortex-A9 processors, Quad-Core Intel Atom CPUs, and programmable FPGAs.

The company offers newer, faster processors in its latest models to suit applications with greater processing requirements, such as a high-speed system inspecting 20 bottles per second. Older smart camera models with single-core processors suit slower applications, such as a standard machine vision system inspecting three parts per second.

Developing smart cameras with image sensors up to 12 MPixels in size, Tattile (Mairano, Italy; www.tattile.com) offers the S12MP camera based on the 12 MPixel CMV12000 CMOS image sensor from ams. The camera also features a dual-core Cortex-A9 667MHz CPU and Xilinx (San Jose, CA, USA; www.xilinx.com) Zynq 7030 programmable FPGA, GigE port, 300 fps frame rate, Linux, and a GigE server on-board for GigE Vision protocol management, while targeting applications such as semiconductor, pharmaceutical, and surface inspection, as well as measurements and analysis.

Available in color and monochrome options and GigE ports, National Instruments’ (Austin, TX, USA; www.ni.com) smart cameras offer image sensors ranging from 0.3 MPixel up to 5.3 MPixels, Intel Celeron N2807 processor core, and can include built-in digital I/O and industrial communication options for communication and integration with industrial automation devices including programmable logic controllers (PLC), human machine interfaces (HMI), robotics, sensors, and industrial machinery.

Companies developing smart cameras with 5 MPixel image sensors include Datalogic (Bologna, Italy; www.datalogic.com) in its T49 smart camera, Omron Microscan (Renton, WA, USA; www.microscan.com) in its MicroHAWK series, Soliton Technologies (Bangalore, Karnataka, India; www.solitontech.com) in its NUERA camera series, Teledyne DALSA (Waterloo, ON, Canada; www.teledynedalsa.com) in its BOA2 XA5 camera, Vision Components (Ettlingen, Germany; www.vision-components.com) in its VCSBC nano Z-RH 0012, VCSBC4012 nano, VCSBC6212 nano, and VCSBC6212 nano, VC4012 nano, and VC6212 nano cameras, and Banner Engineering (Minneapolis, MN, USA; www.bannerengineering.com) in its VE205G1A model.

Embedded smart cameras

While not traditional smart cameras, embedded vision cameras with MIPI interfaces offer some of the same capabilities, including image pre-processing. Basler (Ahrensburg, Germany; www.baslerweb.com) for example, offers a proprietary BCON for MIPI interface designed specifically for systems with MIPI CSI-2 connections. Image pre-processing takes place in the image signal processor (ISP) of the Snapdragon system-on-chip (SoC) from Qualcomm (San Diego, CA, USA, www.qualcomm.com). Basler developed these products, according to Dr. Thomas Rademacher, Product Manager for the embedded vision camera modules, as a compact and cost-saving system to replace classic smart cameras.

The company’s dart camera with BCON for MIPI interface supports Snapdragon SoCs and is available in two models—with an S-Mount or with no mount—both of which use the 5 MPixel color AR0521 CMOS sensor from ON Semiconductor and achieve a frame rate of 60 fps.

Models with in-camera image pre-processing based on the BCON for MIPI interface include four camera modules (color with bare board or S-Mount options) with either 5 MPixel (60 fps) or 13 MPixel (30 fps) CMOS image sensors from ON Semiconductor. As a processor-specific driver is needed, these cameras come with a ready-to-use driver package for NXP’s i.MX 8 applications processor series.

Basler also developed a proprietary BCON for LVDS (Low voltage differential signaling) interface, allowing the dart camera modules to connect directly into an SoC or FPGA, offering a lean setup for developing stereo vision or recognition algorithm-based applications, according to the company. Monochrome and color models of the camera use the 1.2 MPixel AR0132 CMOS image sensor also from ON Semiconductor, the 2 MPixel Teledyne e2v (Chelmsford, Essex, UK; www.teledyne-e2v.com) EV76C570 CMOS image sensor, and the 5 MPixel MT9P031 from ON Semiconductor.

LX VisualApplets cameras feature GigE Interface and offer 2D, 3D, and cameras with JPEG image compression. The 2D cameras have color or monochrome sensors, range in size from 2 to 20 MPixel with CMOS image sensors from ams and reach frame rates of up to 338 fps using VisualApplets. The 3D cameras have monochrome 2 and 12 MPixel CMOS image sensors from ams and use laser triangulation based on image preprocessing in VisualApplets to capture 3D data. LX cameras with JPEG image compression have monochrome and color CMOS image sensors from ams and ON Semiconductor, ranging from 2 to 25 MPixels, and reaching frame rates up to 140 fps.

Equipped with the Intel Movidius (San Mateo, CA, USA; www.movidius.com) vision processing unit (VPU), the MV-IM1013-12MWG smart code reader (Figure 4) from HIKVISION (Hangzhou, China; www.hikvision.com) features a monochrome 1.3 MPixel PYTHON 1300 CMOS image sensor. The camera offers code-reading algorithms for formats such as QR, Data Matrix, and others.

“Capable of deep learning-based barcode location and identification,” says Paul Qiu, Overseas Product Director, Machine Vision, HIKVISION, “the code reader’s VPU greatly improves decoding speed and recognition rate, even on damaged, low-contrast, and light-reflecting codes.”

Five models in the X86 smart code reader family use the PYTHON 1300, PYTHON 2000 (2.3 MPixel), and PYTHON 5000 (5.3 MPixel) CMOS image sensors, and the 8.9 MPixel IMX267 CMOS image sensor from Sony (Tokyo, Japan; www.sony.com). All models feature monochrome sensors and GigE interface. In its X86 open platform, the company offers eight models based on the PYTHON 1300, PYTHON 2000, PYTHON 5000, and the Sony IMX178 STARVIS 6.44 MPixel CMOS image sensor.

Dahua Technology USA (Irvine, CA, USA; www.us.dahuasecurity.com/machine-vision) also offers smart cameras equipped with the Intel Movidius Myriad 2 VPU in its 5000 series cameras. Available with Ethernet or USB interfaces, the cameras employ PYTHON 300 (0.3 MPixel), PYTHON 500 (0.5 MPixel), PYTHON 1300 (1.3 MPixel), PYTHON 2000 (2.3 MPixel), and PYTHON 5000 (5.3 MPixel) CMOS image sensors from ON Semiconductor, and two models use the 6.44 MPixel Sony IMX178 sensor.

These cameras feature algorithms for barcode reading, vision localization, template matching, measurement, and optical character recognition, and with the VPU, the computing performance of the camera improves while also reducing power consumption, according to the company.

In addition to these cameras, Dahua offers a range of smart cameras based on CMOS sensors that range from 0.3 MPixel up to 6.2 MPixel sensors, and a series of X86 smart cameras and code readers, both of which have the same sensor lineup.

Not to be outdone, IDS Imaging Development Systems (Obersulm, Germany; www.ids-imaging.com) has put a new spin on smart cameras with its IDS NXT series (Figure 5) that lets developers quickly set up various image processing tasks by uploading through Smart GenICam new camera functions as a vision app on the camera, according to the company.

The firmware integrates the application code into the camera’s functions. Control and function parameters, and the results of the self-programmed application, are available in standardized form through the device-specific XML description file of each GenICam-compliant application. Third-party software such as HALCON or LabVIEW from National Instruments (Austin, TX, USA; www.ni.com) use these features as if they were part of the standard functionality of the cameras.

Currently, the company offers the rio and rome cameras—which have the same specifications aside from IP65/67 protection class on the rome—are currently available with a range of CMOS sensors, and industrial M12 connectors, C-Mount lens mount, GigE interface, Power over Ethernet, Serial RS232 interface, REST Web interface, dual-core CPU for onboard processing, FPGA, and HALCON runtime license. Additionally, a specially-developed artificial intelligence (AI) vision app turns the cameras into inference cameras with an integrated FPGA-based artificial neural network accelerator.

“IDS NXT aims to create industrial cameras as versatile as smartphones,” says Kai Hartmann, Product Manager, IDS NXT at IDS Imaging Development Systems. “A single device suits many different imaging tasks by installing and switching vision apps – and artificial intelligence takes this even further.”

He continues, “Our IDS NXT rio & rome models provide the possibility to upload pre-trained neural networks onto the camera, enabling it to solve image processing tasks that simply weren’t possible before.”

Vision Components also recently launched a series of embedded vision camera modules with MIPI CSI-2 interface (Figure 6). These cameras feature CMOS image sensors from OmniVision (Santa Clara, CA, USA; www.ovt.com) and STARVIS and Pregius models from Sony, ranging from 0.4 MPixels to 12.4 MPixels in size. The cameras are compatible with a wide range of CPU boards, as well.

Other offerings

Additional companies offering cameras in the traditional smart camera category with lower-resolution image sensors include Keyence (Osaka, Japan; www.keyence.com) with its IV series; SICK (Waldkirch, Germany; www.sick.com) in its CVS, Inspector, midiCam, picoCam, IVC-2D, and Lector series; ADLINK Technology Inc. (New Taipei City, Taiwan; www.adlinktech.com) with its NEON x86 quad-core seriess; and MATRIX VISION Oppenweiler, Germany; www.matrix-vision.com) with its mvBlueGEMINI series of smart cameras.

Increasing needs in large part inspection and the detection of smaller defects in images, as well as higher-speed applications needing greater processing power, led to the evolution of smart cameras—a trend worth monitoring going forward.