SPOTLIGHT: Spectral Imaging Sorts 'Sugar-End' Defects
Markus Burgstaller and Marcus Groinig
For many vision-system applications, capturing and processing images of products in the visible region of the electromagnetic spectrum is sufficient to ensure that measurement and inspection tasks can be carried out accurately and efficiently. However, some applications are more demanding—inspecting the properties of food is one of them. Simply capturing images of products in the visible spectrum may not provide enough data for a system to inspect or classify food products.
Recognizing the need to perform a more sophisticated form of image capture and analysis, many developers of food inspection systems are now turning to multispectral imaging systems to help them with the task.
This was the approach taken by EVK when it was chartered with helping to develop a system with machine builder Insort to inspect the quality of potatoes, specifically determining whether or not they had been afflicted by what are known as "sugar-end defects."
Sugar-end defects are a widespread growth disorder on Irish potatoes in which high levels of reducing sugars such as glucose and fructose and starch concentration are found at one end of the tuber. As a result of plant stress during the growing season, the potatoes accumulate excess sugar during storage. When fried, the sugar caramelizes, producing an undesirable dark brown color at one end and an acceptable golden tan color at the other.
While this might seem a trivial issue, it is a problem that french fry processors abhor because it affects the quality of their finished product. Indeed, it is such an important issue that large fast food companies can return complete shipments to potato processors or even cancel shipment contracts if they identify batches of product with the problem.
One solution employed by potato processors is to cut off both ends of each potato, simply because the defects cannot be detected by the human eye. This approach, however, leads to financial loss since the cut fractions lead to finished products with lower margins. Furthermore, cutting both ends of tubers results in shorter french fries that are of lower quality and price.
Automated inspection
To automate the task of potato inspection while reducing waste, EVK and Insort identified that sugar-end defects are not visible using wavelengths between the recognizable 380–780-nm spectrum but at a wavelength of around 1300 nm. Because of this, a near-infrared (NIR) camera, EVK's Helios-EC3, was chosen to analyze the potatoes for defects.
The Helios camera is mounted on an industrial sorting machine where it captures images of steam-peeled potatoes illuminated by a halogen light as they traveled down vibrating channels at a rate of 1.2 m/s (see Fig. 1). Image data are then processed to identify defective potatoes, which are rejected by the sorting machine using mechanical ejection units.
Since parts of the potato will absorb specific wavelengths in the 1100–1700-nm range according to their fundamental chemical properties, the Helios camera captures spectral images with wavelengths in this range at a rate of 330 frames/s.
In the camera, light reflected from the potatoes passes through an imaging spectrograph that diffracts the light into spectral components after which it is imaged on a push-broom sensor. The columns of the sensor capture the spectral information while its rows capture the spatial information (see Fig. 2a and b).
Images from the sensor are then represented by a three-dimensional hyperspectral datacube with two spatial dimensions (x and y) and a spectral dimension (z). Hence, each image pixel contains a complete reflection or absorption spectrum, while a single plane through the hypercube shows the intensity response or the absorption of the potato at different spectral bands. By analyzing this spectral response, the system can determine if a potato has been afflicted with sugar-end defects.
Spectral characteristics
Before the system can be used to produce pass/fail data, it has to be taught which spectral characteristics of the potatoes are related to the sugar-end defects. To do so, EVK spent one year capturing numerous images of Russet Burbank potatoes with the Helios camera system. These images were then preprocessed in the same fashion that they would be when the camera was fitted to the production machine. Finally, the potatoes were fried to highlight the sugar-end defects and images of them compared with the spectral data captured by the camera.
To determine the relationship between the wavelengths and intensity of the captured spectra to the specific chemical characteristics of the potato—namely the peel, potato, and sugar-end defect—a multivariate data analysis was performed on the spectral data (see Fig. 3). This was accomplished using EVK's EC3 Config, a software suite that analyzes and classifies the content of each pixel in the hypercube.
A chemical-to-color transform was developed to reduce the amount of spectral data generated by the hyperspectral imaging system. This transform associates colors to the pixels in the hyperspectral image of the potato, highlighting three specific areas that relate to sugar-end defects.
This spectral-to-color transformation algorithm was used in the production setup to produce similar pseudo-colored images from the hyperspectral data of the potatoes as they moved through the sorting machine. By processing this color data in-line, rather than the hyperspectral data, the system then recognizes specific properties of the potato at a high throughput rate.
Data reduction
Scanning an object with a hyperspectral imaging system creates a large amount of data since every single spatial point has spectral information associated with it. Because of this, the Helios camera first preprocesses the data to reduce their size. Image data are streamed into field-programmable gate arrays (FPGAs) in the camera at a rate of 330 frames/s where they are first normalized to correct for the intensity variations of the images that are caused by different geometric characteristics of the potatoes.
This technique eliminates any interference due to light from the environment and achieves a consistency in dynamic range across all the spectral bands. Then, a foreground-background segmentation algorithm is applied to remove the background from the image (see Fig. 4).
Finally, spectral information is transformed into an RGB color space by the color transformation algorithm. Each pixel in the resulting image has color information associated with it that relates to the chemical composition of the potato at that specific point. The image itself highlights sugar-end defects in red, the bulk of a good potato in green, and its peel in blue (see Fig. 5).
Color data are then processed further by the FPGAs in the camera and used to sort those potatoes with sugar-end defects or potato with peel from those that do not have them. The spatial dependencies of each pixel in the image are calculated according to their color to determine the precise amount of the defect and hence whether or not the machine needs to take any action. No additional computational power is required.
Sorting spuds
To understand how the data from the camera are used, it is important to understand the mechanics of the machine itself. The potatoes are first loaded into a hopper from where they are fed into a row of vibrating V-shaped channels before being transported under the imaging system, which is mounted above them. As they pass through the imaging system, the camera identifies their spatial position in the V-shaped channels as well as their chemical properties.
Should a specific potato be identified as having sugar-end defects or peel associated with it, the camera uses the decision made from the interpretation of the color image of the potato together with its positional information to send a signal across a serial interface to a custom-built intelligent power subsystem. This subsystem—also developed by EVK—is used to drive paddle-like mechanical actuators at the end of the channels to reject any defective potato, or impurities. In doing so, the potatoes are sorted into separate bins according to how they have been classified by the camera.
Seeing red
To view images of the potatoes as they travel down the line, as well as how they are classified, the potato sorting system can be optionally fitted with a Siemens S7 touch-panel user interface that communicates using the GigE interface from the Helios camera.
On the panel, potatoes classified with sugar-end defect are framed in red; those with peel remaining on them are framed in blue. The use of such a system also enables the data from the production line to be logged for quality control purposes. When deployed, the sorting machine can reject potatoes found to have more than 5% sugar-end defects. In doing so, the system can effectively classify 91.7% of sugar-end affected potatoes. In the production environment, the machine is capable of sorting 16 tons of potatoes per hour.
Without the need to teach the system further, foreign objects like stones, bones, glass, plastic, and wood can be also shown in different colors because of their different spectral characteristics.
Having successfully put the sorting machine to work at Frish and Frost Nahrungsmittel, a leading German food processor, the engineers at Insort have refined it, creating a chemical imaging separator capable of detecting sugar-ends on raw potatoes and detecting and rejecting potatoes with other defects such as green and rotten spots.
Markus Burgstaller is head of research and development and Marcus Groinig is research engineer at EVK DI Kerschhaggl (Raaba, Austria).
Company Info
EVK DI Kerschhaggl
Raaba, Austria
www.evk.co.at
Frish and Frost Nahrungsmittel
Hollabrunn, Austria
www.frisch-frost.at
Insort
Feldbach, Austria
www.insort.at
Siemens
Munich, Germany
www.automation.siemens.com