BIOMEDICAL RESEARCH: Image processing and neural networks assist the blind

Age-related macular degeneration and retinitis pigmentosa are two of the major causes of loss of eyesight, leaving patients with difficulty reading, sewing, or discerning faces. In severe cases, these patients may lose their vision entirely.

To aid those suffering from such conditions, Eng. Enas Elbarbary of VACSERA (Cairo, Egypt; www.vacsera.com), Ass. Prof. Hussam Elbehiery of the Air Defense Forces (Cairo, Egypt), and Prof. Abdelhalim Zekry and Prof. Adel Elhennawy of Ain shams University (Cairo, Egypt; www.shams.edu.eg) have developed a portable system, known as the Blind Assistant, to help the visually impaired recognize their surrounding objects and avoid dangerous situations. This automated system consists of a portable camera interfaced to a mobile computer that uses image-processing techniques and neural networks to detect objects and their colors and provide path-planning aids.

Using the MATLAB Image Acquisition Toolbox from The Mathworks (Natick, MA, USA; www.mathworks.com), images from the camera are first captured as objects where their settings can be edited and stored. To allow surrounding objects, characters, traffic signs, and currency to be interpreted, the system uses The Mathworks’ Neural Network Toolbox for pattern recognition, a separate algorithm to determine the color of an object, and template matching for path planning. By employing a sound card in the portable computer, specific instructions about an object, its color, and its distance are then relayed to the user.

To locate objects and determine the distance between them, the Blind Assistant system uses template matching to predict distances between objects and the number of steps required to reach them.

To locate objects, the Blind Assistant system uses template matching and predicts the approximate number of steps required to reach them.

“For object recognition, neural networks must be adjusted or trained based on a comparison of the output image and the target image so that a particular input leads to a specific target output. Because of this, developers deploying neural networks need not develop algorithms to perform specific tasks,” Elbarbary says.

Using a number of previously acquired images, Elbarbary trained the system’s neural network with objects consisting of characters, objects, currency, and traffic signs. To determine the color of an object, the average RGB values in the image are computed and compared with previously stored colors in a color library. While object recognition was achieved using a predictive feed-forward neural network, template matching is used for objects location determination. The approximate distance between the blind and the objects as well as the number of steps required to reach them are predicted (see figure).

“By using neural networks for object recognition, features within specific images can be automatically trained and used to match future images as they are captured. Since training time is not dependent on processing time, comparing a newly captured image to the trained model, character recognition, and object recognition are relatively fast,” Elbarbary says. By increasing the number of these trained models, the likelihood of finding the captured object using the neural network is increased.

To determine the number of steps needed to reach an object, an approximate method is used to predict them from captures. This is more compute-intensive than using a neural network and increasing the number of templates needed to determine different object orientations, scale, and accommodate changes in illumination or increases in the processing time required.

The researchers have tested the Blind Assistant system in both an office and exterior street settings using both neural networks and template matching for object recognition and for path planning. To image characters, objects, currency, and traffic signs, the average recognition time (per character) was approximately 0.5 s. Using the artificial neural network for object recognition took approximately 0.9 s. However, when template matching was used for path planning, the average processing time increased to approximately 1.7 s for long distances and 7 s for short distances.