Machine Vision System Monitors Greenhouse-Grown Specialty Crop
Researchers at Penn State (University Park, PA, USA) evaluated the performance of a machine vision system, including an AI-enabled image segmentation model, to continuously track the growth of bok choy grown in a greenhouse.
Controlled environment agriculture (CEA)—in which plants are grown indoors under ideal conditions and often in a soilless growing medium—provides a way to harvest vegetables year-round, potentially increasing the volume of food available for an expanding worldwide population.
To maximize crop yields, and thus the ROI of controlled environment agriculture, growers monitor the health of individual plants and environmental conditions throughout the plants’ growth cycle. But managing crops at the plant level manually on a continual basis is time-consuming and requires people with specialized training.
Related: Machine Vision System Tracks Vineyard's Crop Yields
This is why growers are exploring ways to adopt machine vision and other technologies to automate monitoring processes. The goal is not only to automate tedious tasks but to increase the frequency and accuracy of data on individual plants, improving data quality, estimates of plant yields, and crop outcomes.
“By integrating digital cameras into agricultural IoT systems, an expanded array of plant growth parameters can be continuously monitored and recorded, alongside environmental data. This integration is particularly beneficial in CEA systems, where computer vision can continuously and consistently monitor the dynamic growth patterns of plants,” write Chenchen Kang, Long He, and other authors from the Penn State College of Agricultural Sciences in a recent article in Computers and Electronics in Agriculture.
The researchers work out of Penn State's Fruit Research and Extension Center (Biglerville, PA, USA). Led by He, associate professor of agriculture and biological engineering, the research group focuses on automated precision agriculture. They have developed automated solutions for crop picking, tree pruning, green fruit thinning, pollination, orchard heating, pesticide spraying, and irrigation.
Deploying Image Segmentation
The researchers evaluated an image segmentation model that they say is ideal for monitoring plants. First, it is a one-shot model, so extensive labeling of training datasets isn’t necessary. Second, the model is recursive, meaning that it uses segmentation results from previous sets of images as prompts for segmentation of a current set of images.
Related: Researchers Demonstrate New Framework for 3D Imaging
This approach overcomes the challenge many so-called one-stage segmentation models face: detecting individual plants in images in which leaves from neighboring plants overlap, which complicates “the segmentation process due to the challenge of distinguishing individual plants,” they write.
They say the goals of the study were to build an IoT growth monitoring system that captures top-view images at pre-determined times and an image segmentation model that distinguishes individual plants even though their leaves overlap with those of neighboring plants.
Setup of the Machine Vision System
The researchers set up their machine vision system in a greenhouse.
To collect color images and 3D point cloud data, they deployed a ZES X Mini global shutter stereo camera with integrated lens from Stereolabs (San Francisco, CA, USA), which they positioned 0.84 m above the plants. The camera collected images at 60 fps with a resolution of 1920 x 1200 pixels and a depth field of 0.15 to 12 m. The images and point cloud data were stored on a Jetson Orin Nano board from NVIDIA (Santa Clara, CA, USA).
A total of 108 images, depicting various stages of plant growth, were collected over a 27-day period. Of those, 81 images were used in experiments in which researchers compared results of the recursive image segmentation model with two versions of the Segment Anything Model (SAM), a type of transformer model often used for image segmentation tasks, the researchers write. They developed their segmentation model based on SAM.
They trained and validated the models using a local PC.
The Image Segmentation Model’s Performance
They found that both versions of SAM began to fail in segmentation tasks once the plants’ leaves overlapped. Meanwhile, the recursive segmentation model “maintained superior performance throughout the experiment,” the write.
Related: John Deere Unveils New Autonomous Machines
However, the model isn’t perfect. Errors in segmenting previous sets of images would negatively impact subsequent segmenting tasks. In addition, the model’s reliance on earlier images also could cause problems in identifying individual plants in future sets of images in which changes, such as lighting conditions, altered the appearance of a plant’s features in the images.
Related: Advanced Visualization Method Improves Understanding of 3D Point Cloud Data
Overall, the system shows promise for improving plant yields in controlled environment agriculture. “The system’s ability to deliver accurate segmentation results enables the extraction of quantitative features such as plant color, shape and size from digital images, which can then be correlated with manually measured growth parameters, including fresh weight, dry weight, and leaf area. Such correlations can support the development of predictive models for plant growth and yield estimation," they write.
About the Author
Linda Wilson
Editor in Chief
Linda Wilson joined the team at Vision Systems Design in 2022. She has more than 25 years of experience in B2B publishing and has written for numerous publications, including Modern Healthcare, InformationWeek, Computerworld, Health Data Management, and many others. Before joining VSD, she was the senior editor at Medical Laboratory Observer, a sister publication to VSD.