Software transforms images and video into 3-D

An engineer from the University of Illinois at Urbana-Champaign (Champaign, IL, USA) working with software developers at Microsoft Research has developed an algorithm that can automatically transform 2-D single images, as well as 2-D video sequences, into 3-D.

Kevin Karsch and his Microsoft colleagues first used a Kinect-based system to collect a large dataset containing stereoscopic videos with known depths.

Given an input image, the algorithm they developed then finds matching candidates in the database and warps them to match the structure of the input image. A global optimization procedure is then used to interpolate the warped images, producing per-pixel depth estimates for the input image.

On single images, a label based smoothing procedure is then used to improve the depth estimates. On video sequences, a motion flow technique is used.

The researchers demonstrated that the software could produce 3-D images of both indoor and outdoor scenes from 2-D images, and have also created a 3-D version of scenes from the feature film Charade.

Readers can download both the Matlab implementation of the algorithm and the training data from the Microsoft website here.

The researchers have also written a detailed technical article describing the software. Entitled "Depth Extraction from Video Using Non-Parametric Sampling," it can be downloaded from researcher Kevin Karsch's homepage here.

Recent articles describing developments at Microsoft Research that you might also find of interest.

1. Microsoft's Kinect helps keep surgery sterile

Engineers based at Microsoft Research (Cambridge, UK) have developed a system based on the Microsoft Kinect that allows surgeons to manipulate images in the operating theater.

2. Wearable camera-projector combination creates interactive surfaces

A wearable projection system developed by researchers at Microsoft Research (Redmond, WA) and Carnegie Mellon University (Pittsburgh, PA) turns pads of paper, walls or even hands, arms, and legs into graphical, interactive surfaces.

3. 3-D image reconstruction uses multiple structured-light cameras

Microsoft researchers, working with engineers at the University of Science and Technology of China and Texas A&M University (College Station, TX, USA) have developed new software that has enabled them to reconstruct a 3-D scene from multiple structured-light based cameras.

Vision Systems Design magazine and e-newsletter subscriptions are free to qualified professionals. To subscribe, please complete the form here.

-- Dave Wilson, Senior Editor, Vision Systems Design