Machine Vision System Ensures Accurate Girl Scout Cookie Shipments

A smart pallet solution designed by ADLINK enabled the sole distributor of Girl Scout cookies in the United States to double its shipping volume while decreasing error rates. The solution may also provide a measure of security in the face of workforce unavailability owing to the COVID-19 crisis.

Thanks to the pandemic, the long-standing tradition of Girl Scouts standing at the entrances of big box stores behind tables of stacked cookie boxes has been shut down in 2020. Instead, a system for online ordering and deliveries gets the cookies to individual Girl Scout troops and councils.

Evans Distribution Systems (Melvindale, MI, USA; www.evansdist.com), a third-party logistics and supply chain solution company, is the sole online fulfillment provider in the United States for these cookie orders. The popularity of Girl Scout cookies makes this an enormous task. Picking and packing the correct cookie flavors for every shipping box that leaves the Evans warehouse in Michigan, USA can prove challenging.

Related: 3D vision systems provide quality control at copper refining plant

Stephen Ruch, Vice President of Warehousing Operations at Evans, says the company first met with ADLINK (Zhonghe, Taipei, Taiwan; www.adlinktech.com) representatives and expressed interest in their smart pallet system in September 2019. ADLINK had the hardware installed and running two days after arriving at the Evans warehouse in November 2019, according to Ruch.

ADLINK then spent several days training the system’s artificial intelligence (AI) software to recognize by box color all the Girl Scout cookie varieties shipped from the warehouse (Figure 1). On the very first day of the system’s operation, it achieved a 90% accuracy rate. This percentage increased to approximately 99.8% with subsequent training over 4 to 6 weeks, and fine tuning of the vision system’s lighting, according to ADLINK

The primary components of the smart pallet system consist of two cameras mounted on an extruded aluminum frame, an ADLINK Edge AI Gateway computer, and two SP-E-500IIB LED lights from Genaray (www.genaray.com).

A Basler (Ahrensburg, Germany; www.baslerweb.con) Ace acA1300-75gc model mounts directly above the conveyor and points downward to scan the contents of each shipping box passing below. The camera features 1.3 MPixel resolution, 4.8 µm pixel size, GigE interface, and a Basler C125-0418-5M C-Mount lens.

A second camera mounts at an angle and registers the tracking number on each box, verifying the contents against the warehouse’s database. This USB3 camera, the Basler acA3088-57um, features 6.4 MPixel resolution, 2.4 µm pixel size, and a Basler C125-0818-5M C-Mount lens.

ADLINK chose both cameras on the basis of cost and the company’s prior experience with the cameras. The cameras connect to an ADLINK EOS-iXM43H-P5000 computer with a 6th Gen Intel Core i7-6700 processor, NVIDIA (Santa Clara, CA, USA; www.nvidia.com) Quadro P5000 GPU with 16GB memory, 16GB DDR4 memory, and a 512GB SSD, running Linux Ubuntu 18.04

The computer connects to the warehouse’s network via Cat7 cable and also runs ADLINK Edge IoT and VMLINK software. Edge IoT grabs images for box detection and barcode detection and scanning, queries the warehouse database to compare the results of the scan to the expected contents of the box, and determines whether each inspection passes or fails. VMLINK software runs the system’s deep learning models and rules-based, 2D barcode reading functions.

A shipping box passes underneath the cameras once every three seconds and a QMRN-0P-0F photoelectric sensor from Automation Direct (Cumming, GA, USA; www.automationdirect.com) triggers the cameras to scan the box (Figure 2). If the contents of the box scanned by the first camera and analyzed by the AI software match the contents specified by the tracking number, the box passes inspection. It then goes to the void fill station for the addition of packing material, then to the tape machine that seals the box, and directly onto the parcel trailer. If the box fails inspection an employee removes the box from the line, puts the correct contents into the box, and then returns it to the line.

Originally, a green light and a red light hung off the aluminum frame and flashed to indicate good or bad inspection results. Evans found that the lights dazed employees, so ADLINK added an ASUS (Beitou District, Taipei, Taiwan; www.asus.com) VT229 touch screen monitor that hangs off the frame and faces down at employees. Now employees can see when a box arrives at the inspection station and the monitor displays a green or a red dot to indicate a passed or failed inspection. Ruch says this has also made it easier for employees to identify which box to pull off the line when an inspection fails.

When the inspection system went online in 2019, Evans used the traditional picking process that begins with an employee affixing a shipping label to the side of the shipping box and placing a matching pick ticket onto the inside of one of the box flaps. Employees along the picking line read this ticket to determine which cookies to place in the shipping box. When the shipping box arrived at the manual inspection station, an employee would check the contents of the box against the information on the pick ticket. If packed correctly, the employee would then place the pick ticket into the box.

Related: 3D laser profiler inspects cosmetics

While the current, automated system uses its second camera to read tracking codes and register which cookie varieties belong in each shipping box, the picking routine still involves the use of pick tickets for use by human employees. This created an early challenge for Evans when the vision system deployed because employees were accidentally covering the box contents with the pick ticket such that the cameras could not see into the box. The system also produced errors when employees did not place the box in the correct position on the conveyor.

“The training that had to be done with the people, with the labor market that we had with COVID going on, the company struggled with the need to constantly change the box checkers,” says Ruch.

Once these initial challenges were addressed, Evans notes that even doing twice the amount of volume as the previous year, the package return and reship rate due to incorrect box contents was less than the year before. The line requires fewer employees for box content inspection, as well, freeing up personnel for assignments elsewhere in the warehouse. The company in 2020 expects double the volume of cookie shipments from 2019. Without the smart pallet system, the company would never be able to keep up, says Ruch.

The number of boxes inspected by the system picked up so much that Evans had to replace the processor in the system’s computer, making it faster and capable of keeping up with the flow of inspections.

Sunlight coming into the warehouse can sometimes make color identification for the system difficult, as direct sunlight changes how color appears to the camera. ADLINK trained the AI software to account for these changes and Evans plans to build a tunnel over the inspection part of the line, at ADLINK’s recommendation, to provide consistent light conditions regardless of the time of day.

Related: Machine vision battery tray inspection system uses 3D scanning

In the near future, according to Ruch, 2D barcodes containing shipping information for each box will replace traditional pick tickets and tracking numbers. Encoding all shipping information into the 2D barcode eliminates the need for querying the warehouse database with a tracking number for box content verification with the software and can speed up inspection times by almost a full second, says Ruch.

Switching to 2D barcodes will also enable robotic picking applications, says Ruch. Designing a vision system to feed pick commands to robot arms based on reading words off a pick ticket represents a complicated task. 2D barcode readers, on the other hand, can easily read packing information off a barcode and feed this simple information to robot arms programmed with the locations of specific bins for different cookie varieties. Evans Distribution plans for a fully robotic picking line by 2021, an initiative whose value has been reinforced by staffing concerns during the COVID-19 crisis.

With the compressed timeline in 2019 for installing and training the inspection system for cookie box inspections, ADLINK conducted the AI training themselves, says Ruch.  While ADLINK also conducted AI training in 2020, Ruch expects that Evans Distribution will in 2021 conduct model training in-house.

This will in part facilitate Evans’ ability to install the inspection system elsewhere in the warehouse for other products the company ships. Ruch expects that managers will take over AI training by loading into the system images of items to detect and images of good inspection results, made possible via ADLINK’s training module software that supports image drag-and-drop.