Vision-guided robot trims leaves off tomatoes in greenhouse

In order to create anautomated and cost-effective alternative to manually de-leafing tomato crops grown in greenhouses, engineers at Priva developed a vision-guided robot fit with telescopic cutters that autonomously handles the process.

Called the Kompano Deleaf-Line robot, Priva’s automation robot travels on tube rails down lanes in the greenhouse which is populated with tomato plants spaced at intervals on either side of the track. The robot is able to move from plant to plant sequentially and identify and remove leaves from each tomato. In order to identify the small green leafs that need to be removed among the other green leaves in the greenhouse, as well as being able to operate in various lighting conditions, Priva build the system with a pair of stereoscopic cameras.

Twocameras were custom built from two pairs of FLIR Integrated Imaging Solutions (Formerly Point Grey) Chameleon3 cameras. These cameras feature the 1.3 MPixel ON Semiconductor PYTHON 1300 CMOS image sensor, which features a 4.8 µm pixel size and can acquire images at up to 149 fps. The cameras capture a wide field of view from both the left and right side of each of the tomato plants

"To enable each pair of stereo cameras to capture reliable images of the tomato plants regardless of the lighting conditions in the greenhouse, the system employs a Xenon strobe light which illuminates the plant. As the strobe emits light every two seconds, the strobe triggers the stereo cameras to expose images at 30 microsecond intervals. This enables the system to capture a uniform set of images each time," said Dr. Tomas de Boer, the Priva engineer responsible for the design of the system.

As images were captured by the two stereo cameras, they are transferred over USB interface to a PC running the open-source Ubuntu operating system and the Robot Operating System (ROS), which is an open-source software framework hosted by the Open Source Robotics Foundation, Inc. (OSRF) for creating robotic applications across a variety of platforms.

At this point, according to FLIR Integrated Imaging Solutions, custom image processing algorithms running inside OpenCV are used to process the images from both sets of cameras to identify the location of leaves in the tomato plants in a certain height range that was previously defined by the tomato growers. Once the location of the leaves is identified, the software then calculates the exact 3D coordinates of the petioles (the stalks that attach tomato leaf blades to their stems), which need to be cut by the effector at the end of the robotic arm.

Coordinate data generated by the software is passed to the ROS, which transfers the data to set of intelligent servo drives that power motors that drive the telescopic robot arm to the correct location on the plant, where it then cuts the petioles, removing the leaves from the tomato plant. With the cameras being on the same platform as the robotic arm, the cameras move as the robotic arm moves, capturing new sets of images of the plant from a different perspective. From there, additional leaves are identified and removed until the system cannot identify any more leaves.

Priva’s team is currently working with members of the tomato growing consortium in the Netherlands to finalize a pre-production prototype of the system. Depending on the number of robots that are ordered by the consortium, it will then become available for other tomato growers starting in June 2017.

View more information onPriva.
View more information on Chameleon3 cameras.

Share your vision-related news by contactingJames Carroll, Senior Web Editor, Vision Systems Design

To receive news like this in your inbox, click here.

Join ourLinkedIn group | Like us on Facebook | Follow us on Twitter

Learn more: search the Vision Systems Design Buyer's Guide for companies, new products, press releases, and videos