Fiberoptic endoscopy system performs real-time image enhancement

Traditionally,endoscopic vision systems have required standalone video processor units to capture and display image data. As images from a fiberscope are captured, the interstitial spaces between the fibers become more apparent and negatively affect the subject in view. The resulting dark mesh pattern is more noticeable at higher resolutions where the additional pixels more fully capture the distorted light intensities.

Previous systems would remove this noise within the video processor or with a lens and camera specifically matched to the fiberscope. Aminiature camera embedded at the probe tip can be used to eliminate such effects, but these devices do not incorporate a manual eyepiece (allowing usage in case of camera failure) and are more expensive than traditional fiberscopes. “With the advances in computer and camera technology, uncompressed high-definition video capture can now be directly processed and enhanced on a host PC while using the doctor’s current fiberscope,” says Adam Madoian, founder of ThinkTank Technologies (TTT; Port Chester, NY, USA). “As a result, it eliminates the need for an external video processor and simplifies the overall system design.” This is the concept behind TTT’s SweetVision, a video capture system that uses off-the-shelf fiberscopes, cameras, and PCs and then applies algorithmic filtering to enhance the images as they are captured in real time (see Fig. 1).

FIGURE 1. ThinkTank Technologies has developed a video capture endoscopy system, known as SweetVision, that uses off-the-shelf fiberscopes, cameras, and PCs to enhance images as they are captured in high resolution at 27 frames/s.

Designed for use with touchscreen interfaces and tablet PCs, the system allows the enhanced video to be played back, trimmed, managed, and shared for remote viewing and diagnostics. Still images from the video sequence can also be used for generating patient reports. SweetVision can also serve as a telemedicine system, allowing doctors in remote locations to broadcast patient examinations as they occur.

Endoscopes (with fiber bundles ranging from 10,000 to 50,000 fibers) are directly coupled via a video adapter to a DFK 72AUC02 USB color camera from theImaging Source (Bremen, Germany). With a 1/2.5-in. CMOS progressive-scan image sensor from Aptina Imaging (San Jose, CA, USA), 2592 × 1944-pixel images are first binned on the camera to 1024 × 768 resolution and then transferred at 27 frames/s to the host computer over the camera’s USB interface.

TTT teamed withArtemis Vision (Denver, CO, USA) to develop image capture, processing, and display software for the SweetVision system that would perform image filtering.

“Developing the software for the system was performed in two stages,” explains Tom Brennan, president of Artemis Vision. “First, a software algorithm was developed to perform real-time mesh removal to eliminate the dark spacing between the fibers in the captured images. This was then interfaced with a touchscreen graphical user interface (GUI) to allow doctors to save, trim, and play back video at both full speed and slow motion, zoom in and out, and save individual frames.”

Emgu CV, a cross-platform .NET wrapper to Intel’s OpenCV image-processing library, was employed to develop the mesh removal algorithm. This allows OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, and IronPython. The algorithm computes the grid of pixels caused by the multiple fibers used in the fiberscope. Since light falloff from each of these fibers is essentially a Gaussian function, a morphology algorithm can be used to grow the regions of higher illumination and eliminate any falloff in the restored image.

“Third-party image-processing and machine-vision packages proved too expensive to perform what was essentially one specific task,” explains Brennan. “Such third-party packages would not allow the source code to be modified and rebuilt, and alternative approaches, such as writing code entirely from scratch, were impractical,” he says.

FIGURE 2. SweetVision uses a custom GUI that is compatible with touchscreen and tablet interfaces by having a large and easy-to-navigate button layout. This allows the operator to enhance, play back, trim, manage, and share video for remote viewing and diagnostics.

For the frontend, SweetVision uses a custom GUI that is compatible with touchscreen and tablet interfaces by having a large and easy-to-navigate button layout (see Fig. 2). Brennan used C# to allow the GUI to easily interface with the underlying image-processing software. This interface also allows for the playback and filtering of uncompressed and unfiltered video from other sources.

-- By Andy Wilson,Vision Systems Design