AI and OCR Combine for Expiration Date Inspection

A Japan-based foods and daily necessaries wholesaler wanted to automate the process for scanning expiration dates on cartons arriving at distribution centers. Doing so would reduce the staffing required on the line as cartons arrived, allowing employees to be reallocated to other areas of the distribution centers. It also reduced the number of errors recorded and increased the number of cartons that could be processed. The customer’s solution included a combination of artificial intelligence (AI) and optical character recognition (OCR) technologies. Specifically, the expiration date recognition system combines Neurala’s (Boston, MA, USA; www.neurala.com) Vision Inspection Automation (VIA) software and an IHI Logistics and Machinery (Tokyo, Japan; www.ihi.co.jp) designed automation solution.

The Need

The customer sought to reduce the staffing required to scan expiration dates on the side of cartons. Until it installed the IHI solution, the process required personnel to be stationed at a distribution center’s point of entry to manually scan the expiration dates with handheld scanners. It took three hours for two people to receive and check 200 cartons. In addition, error rates were approx. 1/20,000 ~ 1/50,000—mainly human errors. The wholesaler looked to OCR to help reduce the number of errors. Initially, it investigated handheld scanners that employed OCR, but the customer also wanted to reduce the staffing necessary to complete the inspections and using handheld scanners did not accomplish that.

The system IHI provided employs OCR and AI through Neurala’s VIA software. “Previously, using handheld or OCR terminal scans, there was a significant number of misreads by the OCR technology, which resulted in the need for manual intervention and data entry, which increases manufacturers’ costs and reduces profitability,” says Stephen Walsh, vice president of sales, Neurala. “By increasing the read rates, the solution increases automation and maximizes productivity for the manufacturer.” This solution reduced both the error rates and the number of employees required for the task.

The Solution

The system includes a camera and lights to the detect the expiration dates, a barcode reader and photoelectric sensors to identify the items, a PC to detect the expiration dates, and a PLC to evaluate the result of the expiration date inspection and control the total system. Neurala’s VIA software is for the inspection process; the whole control system (software) is by IHI.

Workers manually place cartons on a roller conveyor at the distribution center’s point of entry. IHI proposed full automation with robots, but the customer chose to leave this part of the process manual. At the inspection station, the carton passes a photoelectric sensor from Omron (Hoffman Estates, IL, USA; automation.omron.com) that triggers the beginning of the inspection process. An SR-2000 barcode reader from Keyence (Itasca, IL, USA; www.keyence.com/usa) identifies the carton’s contents. Next, a BFS-PGE-50S5C-C color camera from Teledyne FLIR (Richmond, BC, Canada; www.flir.com/iis/machine-vision) scans the expiration date printed on the side of the cartons. The photoelectric sensor, barcode reader, camera, and lights are all placed next to the line.

The image data is passed via GigE interface from the camera to a any PCs with GPUs, such as GTX1000, 2000, RTX3000 series provided by NVIDIA (Santa Clara, CA, USA; www.nvidia.com) that runs the Neurala AI software. If a carton fails inspection, this triggers a Q series PLC from Mitsubishi Electric (Cypress, CA, USA; www.mitsubishielectric.com) to shunt the carton to a reject line.

Neurala VIA consists of two software programs, Inspector and Brain Builder, which can be installed on any computer with a CPU (Intel 6th Generation or later). It allows for seamless integration on the factory floor: collecting images from GigE Vision cameras and providing outputs over an industrial network (currently supporting Modbus TCP communication) directly to the PLC controlling the applicable machine.

VIA’s role in this process is identifying the location of expiration dates on packages to provide a clearer image to present to the OCR technology. In addition, it also validates that the item presented is a date, as opposed to another numerical value (e.g., SKU). “Neurala trained the AI model using images of packages with the associated objects of interest labeled,” says Walsh. When trained with Neurala’s software, the resultant AI model can now present the OCR with just the area of interest and, because the OCR is more ‘focused,’ it can do a better job reading the data. The logic step Neurala applied adds even more intelligence to make sure the system reads the right data. He adds, “We applied an AI detector algorithm to identify the location of textual and numerical data on packages to facilitate OCR’s ability to make a good reading. Then we applied additional verification and logic to the OCR to heighten the likelihood that the item being read was the intended target and not some other numerical or text-based value.”

The lighting is placed at the camera, but as with many machine vision applications, properly lighting the cartons was a challenge. The lights are 10-W LED lights offering 5000K color temperature and a 120° light distribution angle. In addition, the customer covered the light with a fog-surfaced PET scattering plate to illuminate the surface of the cardboard box evenly. IHI also properly tuned the illumination so every carton could be scanned. If the light was too bright, white cartons could not be scanned.

Items that fail inspection are moved to the reject line, and operators check the packages. The “OK or NG” information is recorded in the warehouse management system. Operators can look at these records on the PC monitor. Cartons that pass inspection continue to move on the main conveyor line to the next operation, which depends on the line.

The systems integrator’s “toolbox” contains various items to use when developing vision/imaging systems appropriate for various applications. In many cases, these tools on their own are not as effective at completing the imaging/inspection task as they are when used in conjunction with other tools. The IHI/Neurala is a good example. “This is a great example of how the combination of two technologies, AI and OCR, can enhance the performance of the solution where each technology in isolation wouldn’t be as effective,” says Walsh. “The same is true for many inspection applications where a combination of traditional machine vision and deep learning can yield significant enhancements in capability to solve problems previously thought impossible. This is a primary reason why Neurala has enabled the deployment of VIA on inexpensive IPCs as a solution that can be retrofitted to existing machine vision inspection systems.”