Innovator's take: 'Free' dimensional video and the future of the moving image medium

The following is a discussion with Matteo Shapira, chief technology officer and co-founder of Replay Technologies. Replay Technologies’ freeD (free dimensional video) captures true 3D scenes that can be tapped to produce any desired viewing angle.

Can you explain how the technology works?

Up until now, video, broadcasting, and film had consisted of cameras capturing 2D image data, which is essentially a sequence of changing flat "pixels" that represent reality. These images are then processed by either post-production facilities, or by ever-growing consumer applications, and end up transmitted and shared digitally.

FreeD technology works by capturing reality not as just a two dimensional, or stereoscopic representation, but as a true 3D scene, comprised of three dimensional "pixels" that faithfully represent the fine details of the scene. This information is stored as a freeD database, which can then be tapped to produce any desired viewing angle from the detailed information.

This enables a new way of capturing reality which us to break free from the constraints of where a physical camera with a particular lens had been placed, to allow a freedom of viewing which has endless possibilities.

To enable the technology, we use synchronized feeds of high resolution video. The video then gets processed using our algorithms to create a 3D database of voxels. After the freeD database is created, we had built an interactive real-time rendering engine that allows viewing of the captured scene from any desired angle (as long as it is within the coverage range of the original sensors).

How did you come up with the idea of "freeD" video?

About a year and a half before the 2012 Olympics games, the three founders of the company gathered to come up with the idea of shooting reality from infinite angles. The catalyst was an opportunity to showcase the technology at the 2012 London Olympics games, which moved the project from the "garage" stage to an actual startup company.

To support the efforts, we brought some of Israel's top notch engineers in the field of computer vision, CGI (computer generated imagery), image processing, and real-time graphics processing unit rendering.

Initially the technology was limited to a very confined (or constrained) space, yet over the next two years we had developed the technology to the point where it can photorealistically represent the entire length of a football field, with accuracy down to a millimeter.

Page 1 | Page 2