Top Crop
Smart camera and custom lighting guide agricultural plant topping and spraying
Environmentalists and farmers alike decry the costs of necessary fungicides and insecticides used in a variety of agricultural crops. Maleic hydrazide (MH) and prime Plus (Prime+) are two such chemicals used to kill the “sucker” or auxillary buds on green tobacco plants. These chemicals effectively control sucker growth but may leave high residue levels on the cured leaf or in the soil. In 1993, R. C. Long of the crop science department at North Carolina State University began exploring new ways to use suckercides such as C6 to C12 fatty alcohols that are contact-based and do not leave undesirable residues.
Figure 1. In farming agricultural crops such as tobacco, suckers are cut off by a topper unit (below) mounted on a high-clearance sprayer that is driven between plant rows (above).
Traditionally, suckercides are applied as a banded spray over the top of the plant row. To be effective, the fatty alcohol suckercides must collect and run down the stalk, making contact with the sucker buds that grow in each leaf axil (junction of each leaf with the stalk). Furthermore, applying these suckercides directly to the broad leaves can sometimes burn holes in the leaves, thereby reducing cured leaf quality and market value.
Manually applying the fatty alcohols directly over the top of the stalk had been shown to be highly effective in killing the suckers and eliminating leaf burn. However, labor costs made this practice impractical. So Long and his team set out to find a way to automate the process by automatically determining the precise location of the stalk of each plant and applying a minimal amount of suckercide as a pulsed spray directly over the stalk. The first application of a suckercide is often combined with “topping” —removing the flower from the top of the plant to allow the remaining leaves to mature before harvest. The cutting topper head is mounted over each row on a high-clearance sprayer. Topper heads are also used to detassel corn in the production of hybrid corn seed.
Therefore, Long’s quest began with the use of a near-infrared through-the-beam sensor, mounted immediately below the topper knife, to locate the plant stalk and to track the plant’s “movement” under the topper head until it reaches a spray manifold (head) and suckercide can be applied as a short burst of spray directly over the stalk. After testing a number of ways of precisely locating the position of the stalk, the final system deployed in 2006 used a smart-camera color machine-vision system that provides better than 95% accuracy for the suckercide application.
Running down the rows
The topper heads are mounted on a high-clearance sprayer so that each topper travels parallel to and over the rows. The topper head in Long’s system has two guide belts whose speed matches the ground speed of the sprayer, thanks to a built-in drive wheel encoder. The belts are used to maintain the plants in an upright position to facilitate locating their position and to ensure the suckercides runs down all sides of the stalk.
As the tall plants are guided into the topper head, a rotary blade, much like a lawn mower blade, cuts off the top of the plant. After the initial topping, farmers make two or three more passes through the field to apply suckercide for season-long sucker control, thereby ensuring that sucker leaves do not restrain growth and marketability of the high-value cash-crop leaves by competing for available nutrients.
Figure 2. To locate each sucker as it passes through the topper unit, a Cognex 552C camera is positioned looking down at plant rows.
Long’s design hoped to use the vision system to identify the stalk location of each plant, track the plant as it moved through the topper, and trigger a spray pulse to apply just enough suckercide to do the job—about 15 ml, compared to approximately 30 ml per plant for traditional applications. This all needs to be accomplished under field conditions at speeds of 2.5 mph and higher, where the 22-in. spacing of the plants translates to roughly two plants topped and sprayed per second.
Near-infrared misses
For its first try at automatically identifying and spatially locating the plant stalks, the group mounted a near-infrared (near-IR) sensor below the topper knife so that the row of plants would pass through the split floor of the topper. Unfortunately, small leaves located just below the cut surface of the stalk could sometimes point toward the on-coming near-IR sensor, thereby prematurely breaking the near-IR beam and causing the small leaf to be mistaken for the stalk and the system to overspray or miss the real stalk.
Long and engineer Wayne Barnhardt decided they needed a new approach capable of greater differentiation and precision, which, in this case, translated to image-processing power. The pair decided to bring on the vision expertise of Jeff and Carolyn Fleming at Industrial Automation Consultants (IAC), a vision-system integrator. “If you don’t know the vision software embedded in the camera, it takes a long time to figure out the slight adjustments you need,” says Long about the decision to bring IAC on-board. “Someone that works with it all the time can really help, especially when you’re trying to solve a difficult problem like agricultural machine vision.”
Smart crop inspection
Long first added Velcro-removable black-cloth side curtains around the sides of the 2 × 4-ft horizontal metal shield that was positioned immediately behind the topper head. This arrangement eliminated both direct and ambient light from impinging on the top of the cut stalks. IAC then positioned a Cognex 552C color CCD smart camera fitted with a Tamron lens and running Intellect image-processing software to focus through a hole in the shield directly over the row as plants passed through the topper and guide belts. “We were trying to keep all the processing in the camera for speed and power considerations,” explains Fleming. “Things happen rather quickly when you are rolling through the field.”
The camera comes with a built-in six-LED white-light source, but IAC determined that the system needed more light to be successful. “We are trying to overcome the reflection of the sun,” says Jeff Fleming. The system uses color to locate the white tissue of the cut stalk against the dark green of the background leaves. Luxeon had just begun offering high-brightness Lumiled LEDs, but no one in the industry had released a high-brightness LED illuminator capable of strobing, so Fleming built his own. “We needed to overdrive the LEDs for short durations to properly capture the desired effect.”
While the camera needed a little help in lighting the stalks, it did have several key features required by the application. “We’d had good experience with Cognex in the past, and the 552C could handle an encoder signal using Intellect software, which was a feature we needed,” continues Fleming.
Cropping and spraying
Although the color and blob analysis used to identify the center of the vertical stalk in the image was done through Intellect, Carolyn Fleming eventually opted to use Java script for camera control for a number of reasons. “The Java scripting in the camera offers much more control over image acquisition and I/O timing,” Carolyn says. Although the plants are spaced about 22-in. apart, leaves from neighboring plants typically fill in the space. IAC used a Telco-Sensor near-IR through-beam type sensor mounted slightly below the top of the cut stalk, just ahead of the camera for triggering.
Figure 4. The tobacco plants enter the topper unit from the right, cross the IR trigger beam, and the 552C captures an image. A spinning blade then cuts off the top of the plant, and the solenoid is triggered to spray the suckercide on the remaining stalk. A Cognex breakout I/O board connects the 552C color camera to the encoder on the topper, the strobe LEDs, and the PLC controlling the sprayer solenoid.
As the topper passes over the row of plants, the trigger signal would rise and fall, depending on the amount of foliage between plants, which resulted in false triggers using the Intellect software. The gain control on the Telco trigger sensor enabled IAC to minimize some of the false triggers. However, Intellect provided Java scripting for more complex controls, and Carolyn wrote a script that would only produce a trigger signal if the sensor output rose above a certain threshold. Also, the camera would only reset its internal encoder count if the vision system issued a pass signal or the trigger sensor threshold was exceeded.
Operating at 20 frames/s, the 552C has a field of view approximately 4 in. wide by 7 in. long, parallel to the plant row. The spray manifold produced an oval dispersion spray pattern approximately 3 × 8 in. during the encoder based “ON” time of the solenoid. IAC developed an algorithm that would provide an x-y location for the stalk in the field of view, which could then be used to fire the solenoid after a number of encoder counts.
The problem was that the number of encoder counts would vary based on where the stalk falls in the field of view. IAC requested a custom function to allow the output timing to be based on a number of encoder counts. Cognex worked with IAC to determine the best approach and provided an I/O function that would allow this to be accomplished. In 2006, this approach led to better than 95% accuracy in applying the suckercide to the top of the cut stalk.
Figure 5. To top off the tobacco plant sucker (top left), Intellect software on the 552C smart camera determines the x-y coordinate of the stalk center within the acquired image, and offsets the spray trigger signal so that the solenoid is activated directly above the stalk (top right). Biodegradable purple dye is added to the suckercide to make it visible for checking the efficiency of the sprayer (bottom).
Although the system was developed for use with tobacco, Long believes the discrimination power and processing speed of the system lend themselves to a range of uses, such as application of chemicals in a variety of crops. “This technology,” asserts Long, “holds the potential for a myriad of benefits for both the environment and agricultural production: directing the application of agrochemicals, increasing efficacy of the application, and reducing the volume of chemical used, production costs, undesirable residues, chemical drift, worker exposure, and the potential for soil and groundwater contamination.”
At present, neither images nor data are stored on the 552C camera. However, Long says the group has explored transmitting data to a separate data logger located near the crop.
null