3D machine vision guides robotic system for logistics e-fulfillment

Turnkey order-picking industrial robot cell is first fully automated system of its kind at DHL

In many traditional e-commerce fulfillment warehouses, depalletizing, picking, and order-fulfillment functions are performed manually. To automate this process and save time and money while doing so, DHL (Bonn, Germany; www.dhl.com), enlisted the help of robotic machine vision specialist Robomotive (Roermond, Netherlands; www.robomotive.nl) for its warehouse location in Beringe, Netherlands, where the logistics company is responsible for the fulfillment and shipping for one of the world’s global manufacturers of ink and toner cartridges.

To accomplish the goal of robotic 3D pick-and-place of boxes from pallets and flow racks to a conveyor, Robomotive developed proprietary object and feature recognition software paired with a Zivid One 3D machine vision camera from Zivid (Oslo, Norway; www.zivid.com).

Figure 1: Robomotive’s system uses a robot-mounted Zivid 3D camera for guidance and a vacuum gripper to move boxes onto a conveyor belt.

“I’ve tested a lot of cameras throughout the years, from gaming cameras to industrial cameras, and worked for a long time with 3D laser triangulation sensors as well,” explains Robomotive CEO Michael Vermeer. “But those sensors cost two-to-three times more and require a lot of effort for calibration. With Zivid, we saw them at VISION 2016 and loved the camera right away when we tested it because it required less calibration, achieved high speeds, and captured great, high-resolution images.”

During operation, online ink orders result in DHL’s warehouse management system sending the order storage rack or pallet location to the robot controller, which initiates commands moving the robot to the correct location. Next, the 3D camera mounted on the robot’s end-of-arm tooling projects a sequence of patterns over the scene using white LED light. By observing the distortion of the light stripes with an integrated camera, the system reconstructs the scene in 3D and produces a point cloud, which Robomotive’s proprietary software analyzes to identify the gaps between the boxes and provide x, y, and z coordinates to the robot. The robot then picks up the box containing the order with a vacuum gripper and moves it to a conveyor belt, where a barcode scanner from Datalogic (Bologna, Italy; www.datalogic.com) ensures the appropriate stock-keeping unit (SKU) has been placed.

Figure 2: The system is used in DHL’s warehouse in The Netherlands to move boxes of printer cartridges.

A feature that gives the system flexibility, according to Vermeer, is the fact that the camera can be integrated onto the end of the robot arm due to its ability to withstand shock and vibration. Zivid’s 3D camera is rated IP65 and is tested according to IEC60068-2-27 standards which includes randomized shocks, sinusoidal vibrations, and frequency sweeps to address robot mounting and harsh work environments.

Related: Multiple 3D methods ease industrial automation applications

“It is important in this application to have the [factory-calibrated] camera attached to the robot instead of having to use multiple cameras, which would require robot-vision calibration and additional costs,” says Vermeer.

In terms of the overall system, the most important thing about it—and why logistics companies like DHL are interested—is because the Robomotive system does not need to train, says Vermeer.

For the other components used in this application, Vermeer explains that—other than the software and camera—the system is not platform-dependent, meaning it can use any robot, PLC, or PC. In this case, the team utilized a Yaskawa Motoman (Miamisburg, OH, USA; www.motoman.com) robot, an industrial PC based on Intel (Santa Clara, CA, USA; www.intel.com) i7 processors with a GPU, and a vacuum gripper.

Figure 3: DHL can reportedly expect about a three-year return on investment for the system, which achieves a speed of 400 picks per hour.

For the system to work, Robomotive defined certain locations by numbering them according to spatial locations—in this case, four pallet locations and around fifty flow rack locations—within the reach of the robot, from which it will pick up boxes and place them onto a conveyor belt. In this case, the robot system has a 6 m reach and 3 m radius.

Related: Pivoting dual-lasers improve 3D bin picking

While the warehouse management system knows which SKU is on which pallet—each pallet and flow rack position contains a single SKU—the robot does not have to know that. It only needs to know where to go when the warehouse management system tells it which position to go to, and the robot knows this because each location has a fixed starting point for vision recognition. For the robot to move a box, it needs to recognize the dimensions of the box with the camera and the box needs to be grippable with the vacuum gripper on the end of the robot arm. Additionally, the vision system recognizes pallets with randomly-stacked boxes it has never seen before. This, according to Vermeer, makes it flexible and unique.

“DHL can exchange products (SKUs) and/or stacking patterns on the pallet location as much as they want, as long as the dimensions and weight are within a certain range,” he says.

For the camera to be able to accurately determine box location and presence, a working distance of about 1200 mm is required, while about 400-500 mm is required for the robot to be able to capture video, see the pallets, identify a separate box, and move it. Accomplishing the two separate tasks requires Zivid’s 3D camera acquiring images at both distances.

A Siemens (Munich, Germany; www.siemens.com) PLC handles control and communication via Profinet and an OPC Unified Architecture (OPC UA) server of all system components.

Figure 4: Outside of the robotic cell, operators can interface with Robomotive’s system and provide instructions on pallet location.

To keep the system running, an operator must remove plastic wrapping from the pallets, refill pallet positions, and reload flow racks. Light curtains in the robotic cell disable the robot so operators can safely replenish the boxes on the pallets for picking. However, users can safely fill the flow racks while the robot is working, so there can always be items being prepared.

Related:Robot-carried vision system inspects engine block surfaces

Challenges encountered during system integration involved communicating with the warehouse management system, according to Vermeer.

“A remote log in to DHL’s ICT [Information and Communications Technology] system from our location requires coordinating with the whole ICT staff and warehouse management system. Most work involved the ‘non-happy flow,’ such as when a position is empty, a barcode cannot be read, a box is dropped. This took months and was our biggest challenge,” he says.

Additionally, the ability to accurately see the boxes to separate them from each other presented an issue. To fix this, Vermeer suggests that they “got everything out of the camera possible, in terms of point cloud quality.”

“This is why 3D cameras are very important here. The better the resolution—Zivid’s camera is HD—the more details on the edges of the boxes we can see, and we needed a lot of details to be able to separate one box from another,” says Vermeer.

DHL can reportedly expect about a three-year return on investment for the system. Going forward, the companies are working together to upgrade this system, including the possibility of deploying the robot on a linear track, allowing it to reach more flow racks and more pallets, with the same camera. Furthermore, given the success of this system, Robomotive is reportedly talking to DHL and other similar big logistics companies about rolling this system out globally.