Memory loss can occur as a result of a number of factors including brain injury or neurodegenerative disorders such as Alzheimer’s disease. While prospective memory (the failure to remember appointments, for example) may affect whether a person can care for themselves, failure of retrospective memory is essential for a person to enjoy any quality of life. “Without memory of shared experiences,” says Steve Hodges of Microsoft Research (Cambridge, UK;research.microsoft.com/cambridge), “it is difficult for a person to maintain any kind of professional, social, or personal relationships.”
While a number of different memory aids such as calendars, watches, and hand-held electronic schedulers may help prospective memory recall, few devices are available to improve recall of past experiences. Because autobiographical memory is thought to be rich in visual imagery, portable image-capture systems can stimulate memory recall. “However,” says Hodges, “traditional digital cameras have a number of drawbacks. First, the patient has to remember to take the picture. Second, stopping to take the picture may break the person’s train of thought, and the resulting photographs may look staged.”
For this reason, Hodges and his colleagues have developed SenseCam, a digital camera that can be worn by those suffering memory loss. Designed to take photographs passively while it is being worn, without user intervention, SenseCam does not have a viewfinder or a display to frame photos. Instead, it is fitted with a wide-angle lens that maximizes its field-of-view to ensure that nearly everything in view is captured.
The camera also contains a number of electronic sensors including light-intensity and light-color sensors, a passive infrared detector, a temperature sensor, and a multiple-axis accelerometer. These are monitored by the camera’s microcontroller so that certain changes in sensor readings can automatically trigger when a photograph is taken.
To capture and process images, the SenseCam camera uses a number of off-the-shelf components in its design. Because the camera is automatically triggered if there is a change in significant light level or temperature, digital sensors are used to monitor these changes. These include an LM75 temperature sensor from National Semiconductor (Santa Clara, CA, USA; www.national.com), the TCS230 color light-to-frequency converter from TAOS (Plano, TX, USA; www.taosinc.com), the SKP-MS401 passive IR sensor from Epson Toyocom (www.eea.epson.com), and a KXP84 tri-axis accelerometer from Kionix (Ithaca, NY, USA; www.kionix.com).
Using a VCR-like interface, stored time-stamped image sequences from the SenseCam can be played slowly (about 2 images/s), quickly (around 10 images/s), rewound, or paused.
“To capture images of significant events but without capturing excessive numbers of images,” says Hodges, “a sensor-based algorithm can beused to trigger image capture when there is a change in light level or temperature.” Also, if the device is stationary, as detected by the accelerometer, and the IR detector senses movement, then an image is captured. Finally, if no movement has occurred for a specific time, then another image is taken.
To capture VGA images, a C328 VGA camera module from COMedia Ltd (Hong Kong, China; www.comedia.com.hk) is used. Like the other sensors used in the design, this interfaces to a PIC 18F8722 6 MIPS microcontroller from Microchip Technology (Chandler, AZ, USA; www.microchip.com). Featuring 128 kbytes of flash memory, 4 kbytes RAM, general purpose I/O lines, and several on-chip peripherals, this microcontroller stores the algorithm that triggers the camera.
VGA resolution images recorded by SenseCam are compressed by the C328 camera module and stored as JPEG files on an internal SECURE DIGITAL memory card. “Because the typical images are about 30 kbytes, about 30,000 images can be stored on a single 1-Gbyte card,” says Hodges. In addition, the memory card stores a log file that records data from the sensors every few seconds. This log file also records the reason for taking each photograph, whether from significant changes in sensor readings or a manual image capture. Time-stamp data are also recorded along with each entry.
Data recorded by the SenseCam can be downloaded onto a PC (see figure). This information can then be manipulated using an image viewer in which images are displayed and can be sequenced to be played slowly, rewound, and paused using a VCR-like interface. It is also possible to import SenseCam image sequences into other applications such asMyLifeBits that allow large number of images to be searched and accessed.
Deployed at the Memory Clinic at Addenbrooke’s Hospital (Cambridge, UK; www.addenbrookes.org.uk), the SenseCam is already helping patients with memory loss. After suffering amnesia from a brain infection, one patient was given a SenseCam and asked to wear it whenever she anticipated an event that she would like to remember. After wearing the SenseCam for the duration of the event, the patient would spend about one-half hour reviewing the images every two days, for a two-week period.
“Without the SenseCam, the patient normally forgot everything about such events after five days or fewer,” says Hodges. “However, during the course of this period of assisted recall, memory for these events steadily increased, and after two weeks of use the patient could recall about 80% of the event in question. Perhaps more remarkable is that following the two-week period of aided recall, the patient appeared to have a lasting ability to recall the event even without reviewing the images.”