3-D IMAGING: Vision-based robot picks plates from pallets
Established in 1853, Jøtul (Fredrikstad, Norway; www.jotul.com) is one of the world’s largest manufacturers of cast-iron stoves and fireplaces. Designed in a modular fashion, the stoves are built from a number of cast-iron plates that compose the side panels and interior of the stove. These are produced at Jøtul and are stored in 1.2 × 0. 8 × 0.5-m pallets that each contains typically 50 cast iron pieces laying in semirandom order.
To eliminate the manual labor associated with unloading these pallets while at the same time increasing throughput, Jøtul enlisted Tordivel (Oslo, Norway; www.scorpionvision.com) to develop a vision-guided pick-and-place system.
“In the design of the system,” says Thor Vollset, chief executive officer of Tordivel, “it was necessary to properly identify hundreds of different types of plates so that they could be unloaded from the pallet in the correct fashion.” Luckily, Jøtul provides each of these plates with four cast identifying markers consisting of concentric circular elements, each of which has a different interior diameter (see figure).
By identifying these elements in 3-D space, both the type and location of the part can be calculated and the robot positioned to pick and place the plate as required.
“In traditional stereo vision systems,” says Vollset, “two cameras are normally used to capture the location of three specific part locations so that its position in 3-D space can be computed.” However, to improve the accuracy of this computation, Tordivel employs three XCGSX97E 2/3-type progressive-scan CCD GigE cameras from Sony (Park Ridge, NJ, USA; www.sony.com/videocameras) that are mounted above the pallet. Then, by comparing the accuracy of the measurements from the three stereo image pairs, the accuracy can be improved.
Further improving the robustness of the system, the geometric data from the located features are compared to a part database. “In this way,” says Vollset, “should any one feature be occluded, the system will not fail.”
To locate these features within each image, image data from each of the three cameras are transferred over GigE interfaces to an Intel multicore CPU. Here, Tordivel’s own Scorpion 3D Stinger software is used to process the images and locate the fiducial markers located on the part.
These images are first normalized and a Canny edge detection filter is applied to account for the variation in illumination and increase contrast. By subtracting the original image from the filtered image, the edges within the images are highlighted. Performing this process on a multicore processor allows this process to be performed in parallel on the three image data sets concurrently and reduces the processing time required to locate the circular markers to less than 1 s.
Once these edges are located in all three images, a RANdom SAmple Consensus (RANSAC) algorithm is used to iteratively determine known circular markers within each image. By locating multiple circular markers in all three images, the location and identity of each part in 3-D can be computed.
After the coordinates of a part are determined, the data are transferred from the host PC to a PLC that is used to control a six-axis robot from ABB Robotics (Cary, NC, USA; www.abb.com). Magnets placed on a custom gripper on the robot are then rotated to fit onto each part. After the part is successfully picked, it is then placed onto a station for transfer to a final assembly station.
By identifying markers in 3-D space, the type and location of a part can be calculated and a robot positioned to pick and place the plate as required.