3-D imaging guides surgical operations

A telerobotic system follows hand movements and displays 3-D images of the operational area as a surgeon manipulates console controls.

By Joe Hallett,Contributing Editor

Computer-aided tools allow surgeons to work through small incisions, which improves precise operations, shortens patient recovery time, and lowers medical costs. Although two-dimensional (2-D) endoscopic vision methods have been used successfully for some time to remove tissue, new three-dimensional (3-D) or stereo vision provides more advantages for reconstructive surgery.

The da Vinci system, developed by Intuitive Surgical Inc. (ISI; Mountain View, CA), combines micromanipulator-controlled surgical tools with 3-D remote vision. It enables a surgeon to perform complex operations while sitting at a console station. Development of this system involved trade-offs in imaging techniques and a force-sensitive human interface. It is the first surgical robotic system to be approved by the US Food and Drug Administration (FDA) for performing minimally invasive surgical procedures in the United States.

The ISI telerobotic minimally invasive approach places the surgical instruments under the complete control of the surgeon. It follows the surgeon's hand movements, rather than relying on a computer for guidance. The surgeon "feels" the motions of robotic tools along with an indication of limits on the range of motion, while simultaneously watching a 3-D image of the operational area. According to ISI's engineering vice president Gary Guthart, "It's hard to get good results from 2-D views. We can do more dexterous procedures with 3-D views."

Henrico Doctors' Hospital (Richmond, VA) is the first medical facility to perform general surgery using the da Vinci system, according to Dr. William E. Kelley Jr., the operating surgeon. The first operation—the removal of a patient's gallbladder—was performed by Dr. Kelley on the day after the system was approval by the FDA last July. First, he tied the patient's gallbladder artery and duct using the system's computer-directed robotic fingers and wrists. Then, while studying 3-D views, he detached and removed the gallbladder from its location on the liver (see Fig. 1).

Inside view
The da Vinci system comprises two major subsystems: a remote 3-D viewer and a telerobotic manipulator (see Fig. 2). Working together, they extend the sight and hands of a surgeon, enabling precise surgical penetration inside a patient's body. The system viewer is built around a custom-designed endoscope—a hollow tube containing glass optics that is inserted into the patient. This endoscope sends detailed 3-D stereo images to a monitor for viewing by the surgeon. Imaging components, illuminators, and circuit electronics typically are placed outside and at the top of the tube .

FIGURE 1. During an operation, the surgeon sits at a console located a few meters from the patient. The doctor controls the surgical tools with robotic-driven micromanipulators and observes the operation through a stereo viewer.

According to ISI's Guthart, the da Vinci system uses a "haptic" interface, characterized by lower frequency and single-point robotic forces compared with those of a traditional tactile (texture-based) interface. He says, "We use master manipulators—tiny robot arms that the surgeon can grab and move—as input devices, but they also are used to measure and display forces and torques. We share a technology approach with those of virtual reality and video games—the force-feedback joystick. But our technology measures and accurately presents the surgeon's motions."

There were problems with endoscope camera technology when da Vinci system developments were started. Weight constraints were significant, and displays were either head-mounted or placed far across the operating room from the surgeon, reducing image quality. Says Guthart, "The 3-D endoscopes available on the market had good resolution and color but didn't have good stereo separation or image registration. Or, they provided good stereo but inferior image quality. The 2-D technique was good for resolution, color, and field of view, but for our work, we needed better stereo vision." Liquid-crystal shutters were considered for the stereo viewer, but were dropped because they produced low brightness and introduced crosstalk defects between the images.

FIGURE 2. Da Vinci system consists of two major subsystems: a 3-D viewer and a telerobotic manipulator. Working together, they extend the sight and the hands of a surgeon and enable surgical penetration inside a patient's body.

The final system configuration solves both design concerns. First, the endoscope is secured by a robotic assembly, rather than being manipulated directly by the surgeon, making the weight of the endoscope unimportant. Second, the surgeon's hands are tethered to the manipulator grips and mandate a fixed position at the control console. This setup immobilizes the surgeon's head in the viewer and directs high-quality 3-D stereo images to the surgeon's left and right eyes (see Fig. 3). This approach eliminated the use of head-mounted displays, which are not popular with surgeons.

Two MCD214 CRT monitors from Barco Inc. (Santa Clara, CA, USA) are positioned in the system viewer about 18 in. from the surgeon's eyes. The monitor placement is carefully matched on the two CRT screens to get good stereo fusion for left and right views. Images are displayed in natural color using standard colorimetry at 60 frames/s. Two-dimensional views are presented on standard TV monitors in the operating room for viewing by other surgical staff members.

"We use video processing to cut down the low-light video noise and to perform image edge enhancement, but no data compression is done in our system," says John Stern, ISI's director of hardware development for the visualization subsystem, including the endoscopic camera, video processor, and display. "Signals are transmitted digitally from the patient to the surgeon's control console, where electronic circuitry converts the interlaced images to progressive (noninterlaced) format, and text and graphics are superimposed," he adds.

Panasonic Security & Digital Imaging Co. (Secaucus, NJ, USA) built the GP-US522 three-chip CCD camera and camera control unit (CCU). The resolution of each CCD chip is 768 x 494 pixels. The TBS24TD synchronizer/processor unit comes from Snell & Wilcox (Santa Clara, CA, USA). The endoscope is custom-built by an OEM medical supplier.

FIGURE 3. The surgeon's hands are tethered to the manipulator grips and the surgeon's head is fixed in position by the viewer. This setup delivers high-quality 3-D stereo images to the surgeon's left and right eyes and eliminates the use of head-mounted displays, which are not popular with surgeons.

"The endoscope is equipped with two independent data channels in one 12-mm tube," explains Stern. "We tried to maximize stereo separation. The result is about what a viewer would see at a distance of 17 to 18 in. when the tissue under study is placed two inches in front of the camera lens. The system also includes a focusing stage under foot-pedal control of the surgeon. The insertable length of the endoscope into an organ is 387 mm (0.16 in.), which is a greater depth than that provided by an average endoscope," he says.

Sterility is imperative in surgical procedures. Some endoscope systems can tolerate sterilization by the autoclave method—a high-temperature process that is easy and fast to perform at hospitals but involves fluid and gaseous chemical baths. However, typical electronic packages are damaged by sterilization methods. As a result, the da Vinci system uses a sterilized plastic drape to cover all the electronic cables during surgery, from the endoscope back to the system controller, which is located about 2 m from the patient.

Illumination is provided by two 300-W xenon illuminators that are connected by fiberoptic light guides to the (endoscope) tip. The illuminators and camera controller are mounted on a movable cart. "We intentionally divert some light power to heat the endoscope tip to prevent fogging," says Stern. These surgical systems are now being deployed throughout the United States.

The next generation of high-technology surgical imaging might be a "distance-surgery" method. This method would enable surgeons to work at computer consoles and perform operations on patients who are located far from the surgeon. "It won't be easy," says Stern, "We have to keep an eye on system latency—from the time the surgeon moves to the time that motion is seen-which takes milliseconds. Approximately a 60-ms latency is tops. This is a limitation for telesurgery," he adds.

Henrico Doctors' Hospital
Richmond, VA 23229, USA
Web: www.henricodoctors.com

Intuitive Surgical Inc.
Mountain View, CA 94043, USA
Web: www.intuitivesurgical.com

Panasonic Security & Digital Imaging Co.
Secaucus, NJ 97094, USA
Web: www.panasonic.com/mv

Snell and Wilcox
Santa Clara, CA 95050, USA
Web: www.snellwilcox.com