Vision system discerns bakery-basket colors

Vision sensor and LED white-light array help identify colored breadbaskets in an automated bakery.

By R. Winn Hardin, Contributing Editor

A North-American wholesale bakery operation was in the process of expanding its manufacturing and warehouse operations and adding automation equipment where feasible. Part of the expansion included an automated sorting line for outgoing stacks of color-coded plastic baskets that are loaded with loaves of bread for distribution and delivery.

Baskets of two different colors—blue and gray—indicate separate product lines for shipping. However, same-color baskets are often placed randomly on the same conveyor line after they are stacked and filled with appropriate bread products. The plant-expansion project included two separate offloading areas: one delivered a particular product line to an automated dolly-loading system, and the other directed a second product line to a manual unloading station farther down the conveyor.

The end user turned to bakery-automation specialists AMF Bakery Systems (Richmond, VA, USA) to provide a new single-stack dolly loader and sorting/conveyor system. Wes Nelson, AMF electrical engineer, initially evaluated a photoeye sensor method to differentiate the color of the two basket types, but stray price stickers and bread labels stuck to the sides of the baskets resulted in unreliable sorting.

After discarding the photoeye sensor approach, Nelson investigated several vision-sensor, or smart-camera, vision systems. Due to limited resolution, black-and-white vision sensors had difficulties in determining the slight hue differences between blue and gray. Consequently, Cross Automation (Richmond, VA, USA) machine-vision specialist and distributor Bryant Butler suggested a low-cost color vision sensor. However, most dedicated color-vision systems require separate software installed on a laptop to configure the system and set up the application. "We wanted this setup to be configurable without a laptop, by a technician, and not by an engineer," Nelson says. "Also, the cost of the software was considered too high."

FIGURE 1. In an automated bakery, a machine-vision system assists in governing the mechanical conveyors and subsystems that color identify and sort stacks of 15 blue or gray breadbaskets. An Omron F400 color-vision sensor, 8-mm color-corrected lens, and associated vision controller unit, along with an RVSI NerLite AR-100 white-light LED array, are connected to an Allen-Bradley SLC-5/04 programmable logic controller that triggers camera imaging when activated by an Allen-Bradley 9000 Photoeye sensor.

With the assistance of Butler, Nelson selected an Omron Electronics (Schaumburg, IL, USA) F400 compact color-vision sensor with an 8-mm, color-corrected, C-mounted lens and an associated sensor controller unit. For lighting, they chose a NerLite AR-100 white-light LED area array illuminator from RVSI/Northeast Robotics (Weare, NH, USA). Both devices are connected to an existing Allen-Bradley (Milwaukee, WI, USA) SLC-5/04 programmable logic controller (PLC). This machine-vision system also aids in governing the mechanical conveyors and subsystems that sort the colored baskets based on the vision sensor's color-imaging capabilities (see Fig. 1).

Color vs. monochrome

Nelson chose the F400 color-vision sensor because it provided multiple I/O connections (as many as 11 inputs and 21 outputs), which could later be used to communicate pass/fail and "system-state" signals across multiple lines to the SLC-5/04 controller (see Fig. 2). It also could be reconfigured using the Omron controller unit instead of a laptop and software. Lastly, the configuration file, including white-balance information, could be uploaded from the F400 vision controller and downloaded to a new controller should the first controller fail or need repair. The other vision systems could not guarantee this option.

FIGURE 2. Stacks of blue or gray breadbaskets come down a long conveyor until they reach the color-inspection stop, where an F400 vision system captures images of the basket stack, determines whether the baskets are blue or gray, and conveys the information to the PLC controller. In turn, the controller instructs the conveyor system to pass the stack straight through to a manual unloading station or else it engages a Cannon 90° transfer conveyor. This transfer conveyor sends the stack onto a 12-ft stack reservoir conveyor to the AMF automated single-stack dolly loader.

"An AMF concern about the color system was that it didn't want to do white balance every time the system was started," says Cross Automation's Butler. AMF's Nelson explains that the Omron system allowed him to "store all the setup data, including the white-balance data, in an ASCII file. If the customer wanted to buy a spare controller, we could download the ASCII file to the spare controller and have it available for swap out. As long as the lighting and aperture didn't change, you were ready to go. We erased everything from the controller memory and downloaded the ASCII file, and the vision system gave us good and bad readings just as it should."

FIGURE 3. Omron F400 color vision sensor is housed in a Hoffman protective enclosure because of its proximity to the 300-lb stacks of fully loaded breadbaskets. Its associated controller unit sits on the bottom shelf. The system PLC triggers the F400 sensor via an output connected to the controller unit. The sensor uses the Omron "color pick-up mode" from the F400 image-processing library to identify up to eight colors.

The new sorting line ties into the end user's existing conveyor system. Stacks of 15 baskets of a single color come down the line filled with products to be loaded onto trucks. Each stack can weigh 300 lb or more. Therefore, the F400 remote-head vision sensor with its color-corrected, 8-mm, C-mounted lens and the vision controller unit are installed perpendicular to the conveyor line to give a good view of the baskets' sidewalls (see Fig. 3). For factory protection, the camera system is housed in a Hoffman (Anoka, MN, USA) protective enclosure with a 3 × 5-in. viewing window.

FIGURE 4. NerLite AR-100 LED white-light area array illuminator comes in 140 × 114 × 22-mm, black anodized aluminum box and provides nondiffused light at 6500K over 96.8 sq mm. Mounted below the camera window at the front of the Hoffman enclosure, it provides a bright and even light across the visible spectrum regardless of ambient conditions and the random opening and closing of warehouse bay doors.

The NerLite white-light LED area array illuminator is installed outside the protective enclosure at a 45° angle because glare precluded putting it inside the enclosure (see Fig. 4). However, the array is attached at the lower front of the enclosure to provide some protection (see Fig. 5). "Testing showed that the LED array provided sufficient intensity for our purposes," says Nelson. "We were concerned about warehouse bay doors opening and closing and causing unstable light conditions. For this vision system, we needed direct light under all conditions. The LED array also comes in a 24-Vdc version, so we could wire it into our dc power supply without installing a separate power supply."

FIGURE 5. AR-100 LED area array illuminator is mounted at the front of the enclosure just below the window at a 45° angle because glare precluded putting it inside the enclosure. The LED array eliminates reflective glare and provides sufficient light intensity despite unstable lighting conditions.

Nelson chose the 8-mm color-correction lens because at a 10-in. standoff it gave him enough depth of field to compensate for the ±1.5-in. variation in basket placement as the baskets moved down the conveyor line.

System integration

Stacks of baskets come down a long conveyor until they reach the basket-color inspection stop. The F400 vision system captures 512 × 484-pixel images of the basket stack at this stop, determines whether the baskets are blue or gray, and conveys this information to the PLC. In turn, the controller instructs the conveyor system to pass the stack straight through to a manual unloading station or else it engages a Cannon Equipment (Rosemont, MN, USA) 90° transfer conveyor. This transfer conveyor sends the stack onto a 12-ft stack reservoir conveyor to the AMF automated single-stack dolly loader. The loader houses the SLC 5/04 PLC and an Allen-Bradley Panelview 550 operator interface for the F400 camera.

As the basket stack comes down the conveyor, it passes an Allen-Bradley Photoswitch 9000 photoeye. Other photoeyes are placed behind the vision system at the 90° transfer conveyor and on the stack reservoir conveyor leading to the dolly loader. If all the photoeyes are clear, the PLC triggers the F400 sensor through a standard output connected to the F400 controller. The sensor uses Omron's "color pick-up mode" from the F400 image-processing library to identify up to eight colors based on hue, saturation, and intensity from 256 possible color variations (8-bit depth color data).

The F400 sends the data back to the PLC along one of two signal/cable outputs: one for blue and one for gray. Another signal communicates the error messages from the F400 to the PLC. Yet another signal conveys a "system busy" message to the PLC while the "color pick-up" function is running. Nelson says he chose to use separate output signals rather than an RS-232 serial connection because it made system programming and trouble-shooting easier for a technician who might not be familiar with the F400's ASCII code. In this manner, one signal is responsible for one action, and errors including wiring failures can be easily isolated and fixed, Nelson adds.

The technician can set the automated system to send baskets of either color straight to the manual unloading station or to engage the 90° transfer conveyor. This conveyor lifts the stack above the constantly moving straight conveyor and sends it to the automated single-stack dolly loader. The PLC also allows the technician to manually trigger the vision system should a problem arise with controller/PLC communications, allowing the conveyor line to operate even under faulty conditions, if necessary.

"The images and diagnostics received from the F400 make this system robust and help to cut down on errors that could foul the system. The vision-system capabilities exceed those of the application, so future expansion and automation are still available," Nelson notes.

Company Info

Allen-Bradley/Rockwell Automation www.rockwellautomation.com
AMF Bakery Systems www.amfbakery.com
Cannon Equipment www.cannonequipment.com
Cross Automation www.cross-automation.com
Hoffman www.hoffmanonline.com
Omron Electronics www.omron.com
RVSI/NER (Northeast Robotics) www.nerlite.com