AI Broadens the Horizon of Machine Vision

Guy Nicholson, Senior Director, Industrial and Commercial Sensing Division, onsemi

In recent years, the use of machine vision systems in industrial processes has increased considerably. Traditionally, these systems are used to recognize basic patterns, which allows them to monitor products being transported on a moving conveyor belt, for example. However, the recent advent of artificial intelligence (AI) and machine learning (ML) promises to bring machine vision systems to the next level in the terms of the value they can bring to industrial applications.

Products rolling off an industrial production line normally have well-defined parameters making this an ideal application for machine vision systems that can be preprogrammed to identify objects or events that fall outside of an expected range. However, they are unable to manage events or occurrences other than those for which they have been configured to recognize. AI and ML will help to address this shortcoming and thereby enable machine vision systems to be employed in much higher value-added activities, simply identifying that if a conveyor belt has come to an unexpected stop or if objects is missing from a production line. This will allow them to support activities across multiple vertical markets.

As costs decrease as accuracy increases, commercially available cloud platforms are increasingly being used to host the AI algorithms that process the vision data gathered by these applications. This means that it is becoming much easier to add AI and ML to machine vision systems that will inevitably become a major part in the ongoing evolution of IoT (and Industrial IoT) devices

Ultra-Low-Power Sensing

The availability of ultra-low-power sensor solutions will be critical to this growth because many applications will require an active vision system that can quickly capture images on demand, combining elevated levels of performance with low power consumption. By processing AI images in the cloud, smaller and more power-efficient sensor platforms can be used, but these will require a robust high-speed data transfer link to the cloud. Bluetooth® Low Energy (Bluetooth LE) technology will be important in providing this connectivity, providing a simple means to wirelessly connect a machine vision sensor platform to remote cloud services via an IoT gateway. In this way, the sensor platform can transfer image data to the cloud platform to be processed by AI algorithms which can then respond (if required) by sending an appropriate instruction back to the sensor or a smart actuator.

Image Capture in Color

Monochrome (black and white) images are sufficient for many machine vision systems to perform basic object recognition, but color images contain much more information and have the potential to bring a new dimension to using AI for scene recognition in many emerging applications. Color provides greater contrast, making it easier to accurately differentiate objects in a scene, which AI systems can more fully exploit. However, capturing color images uses more power than monochrome, which consequently becomes challenging for a battery-powered device that is expected to operate for at least five [CML1] on just a single coin cell. Therefore, when using color images in a machine vision system, power management of the image sensor, control system, and the communications interface is key to maximizing battery life.

Event-Triggered Machine Vision

Instead of streaming image data constantly, event-triggered machine vision captured images only if a predefined event occurs. The conditions for an event are monitored using highly advanced sensors integrated into the camera platform. The conditions that can be monitored using these sensors include motion, temperature, time, humidity, and acceleration. Developers can then use the outputs of these sensors to create complex conditions for event processing. If these conditions for these events are satisfied, the camera triggers an image capture which can then be transferred using Bluetooth Low Energy technology to a smart phone or gateway.

Applications for AI Image Analysis

Event-triggered machine vision makes sense in many applications where a constant stream of image data is neither required nor feasible. One such application is inventory monitoring. Here, an event-triggered camera can be used to monitor stock levels on supermarket shelves and automatically send a restocking alert if gaps are detected on the shelf. As an extension to this application, instead of just sending a generic restocking alert about a specific shelf in a certain aisle, the images can be processed using cloud-based AI algorithms to identify the exact product type that requires restocking. Similarly, this approach can also be applied in a domestic environment to monitor product levels in kitchen cupboards or a refrigerator and be configured to connect with mobile shopping apps to create automatic reordering lists.

By their very nature, long-term storage spaces are often static for extended periods of time meaning continuous video monitoring does not make sense. They are also only intermittently attended by security personnel, which means that in the event of a break-in or some other event that changes storage conditions (e.g., a leak or an air conditioning failure), loss or damage can go undetected for an unacceptable period before remedial action is taken. Here, it makes sense to use a camera that can send an updated image if a change in environmental conditions or unexpected motion is detected. This could also be useful for vacation homeowners who wish to receive updates if similar events are detected at their properties. False alerts can further be avoided by using AI algorithms that have been trained to recognize the faces or emblems of expected visitors such as security personnel.

Nowadays, smart digital meters are used to monitor the status of environmental variables in industrial processes, but there are countless old analog meters still in use. These must be manually read on a regular basis, and unless an operator is in proximity at the time that a measurement moves outside of its expected range, it can go undetected with the potential to cause process downtime or have even more damaging consequences. In this scenario, an event-triggered AI camera can be used to trigger an alarm if the meter reading drifts beyond expected values and send an image of the reading to the process controller (human or machine) to take appropriate action.

Agricultural producers can also use AI machine vision technology in several ways. Apart from monitoring environmental conditions, it can also be used to monitor varying levels of ripeness or detect the presence of disease in fruit or vegetables (through changes in color) or to recognize the presence of unwanted pests (motion) in greenhouses.