BIOMEDICAL RESEARCH: Image processing and neural networks assist the blind

Age-related macular degeneration and retinitis pigmentosa are two of the major causes of loss of eyesight, leaving patients with difficulty reading, sewing, or discerning faces. In severe cases, these patients may lose their vision entirely.

To aid those suffering from such conditions, Enas Elbarbary of VACSERA (Cairo, Egypt; www.vacsera.com), Abdelhalim Zekry and Adel Elhennawy of the Ain shams University (Cairo, Egypt; www.shams.edu.eg), and Hussam Elbehiery of the Air Defense Forces (Cairo, Egypt) have developed a portable system, known as the Blind Assistant, to help the visually impaired recognize their surrounding objects and avoid dangerous situations. This automated system consists of a portable camera interfaced to a mobile computer that uses image-processing techniques and neural networks to detect objects and their colors and provide path-planning aids.

Using the MATLAB Image Acquisition Toolbox from The Mathworks (Natick, MA, USA; www.mathworks.com), images from the camera are first captured as objects where their settings can be edited and stored. To allow surrounding objects, characters, traffic signals, and currency to be interpreted, the system uses The Mathworks’ Neural Network Toolbox for pattern recognition, a separate algorithm to determine the color of an object, and template matching for path planning. By employing a sound card in the portable computer, specific instructions about an object, its color, and its distance are then relayed to the user.

To locate objects and determine the distance between them, the Blind Assistant system uses template matching to predict distances between objects and the number of steps required to reach them.

“For object recognition, neural networks must be adjusted or trained based on a comparison of the output image and the target image so that a particular input leads to a specific target output. Because of this, developers deploying neural networks need not develop algorithms to perform specific tasks,” Elbarbary says.

Using a number of previously acquired images, Elbarbary trained the system’s neural network with objects consisting of characters, objects, currency, and traffic signals. To determine the color of an object, the average RGB values in the image are computed and compared with previously stored colors in a color library. While object recognition was achieved using a predictive need-forward neural network, template matching aided the prediction of distances between objects and the number of steps required to reach them (see figure).

“By using neural networks for object recognition, features within specific images can be automatically trained and used to match future images as they are captured. Since training time is not dependent on processing time, comparing a newly captured image to the trained model, character recognition, and object recognition are relatively fast,” Elbarbary says. By increasing the number of these trained models, the likelihood of finding the captured object using the neural network is increased.

To determine the number of steps needed to reach an object, template matching is used. This is more compute-intensive than using a neural network and increasing the number of templates needed to determine different object orientations, scale, and accommodate changes in illumination or increases in the processing time required.

Elbarbary and his colleagues have tested the Blind Assistant system in both an office and exterior street settings using neural networks for object recognition and template matching for path planning. To image characters, objects, currency, and traffic signals, the average recognition time (per character) was approximately 0.5 s. Using the artificial neural network for object recognition took approximately 0.9 s. However, when template matching was used for path planning, the average processing time increased to approximately 7 s.