High-speed imaging: Vision system speeds high-speed image capture, review and exploration

In military, aerospace, life-sciences and automotive applications, it is often necessary to capture sequences of high-speed images for analysis. In automotive applications, for example, the dynamics of airbag deployment are regularly captured with high speed cameras.

Such high speed captures can accumulate vast amounts of data in a short duration. Similarly, security and surveillance systems often capture equally large amount of data, simply over longer time periods. In both cases, the user is faced with examining oceans of data, and if one is interested in only a small subset of the overall content – perhaps small time windows around particular key events – the process can be daunting.

"Often," says David Durlach, Founder and Director of Temporal Innovations (Somerville, MA; www.temporalinnovations.com), "initial perusal of content is accomplished using software controls similar to those found on a VCR or DVD player. This type of user interface may be adequate when the amount of data is modest in size. However, when the amount of content is enormous, and the events of interest relatively short in duration, such controls are far from ideal.

Further, Durlach notes, "Images captured for motion analysis are rarely best understood through simple 'playing'. It is typically much more informative, and quicker, to explore imagery at changing speeds continuously adjusted to match the timescale of events in view at the moment."

Because of this, Temporal Innovations has developed its FrameGlide|Spin Browser system that enhances the user's experience when navigating large amounts of image data.

The hardware element, the Spin Browser dial, is an industrial grade USB peripheral control wheel. Using the Spin Browser, field-adjustable exponential parameters allow operators to traverse content quickly and accurately over timescales spanning six orders of magnitude. As the velocity of the wheel increases, so too does the number of frames per degree of rotation.

"Because we eliminate buffering, and because the latency of modern hard drives is under 1/30s," says Durlach, "the system allows users to seamlessly scroll through Terabytes of image data, with no temporal stutters. This is true at any speed - from stop frame to millions of times normal rate - backwards and forwards, whether or not viewing contiguous frame paths."

Furthermore, sequences that contain events of interest spanning different time scales may be interactively traversed at whatever speed best highlights the feature of interest. "For example," says Durlach, "consider troubleshooting a packaging operation that involves both a bottle cap applicator and a conveyor belt that brings the bottle to the applicator. In that case, when examining the cap applicator, millisecond time scales need to be reviewed. When examining a conveyer belt for large-scale oscillations, one second time scales may be more appropriate. The FG|SB system handles such situations, without requiring the operator to take their eyes from the subject matter."

As an enhancement to its core technology, Temporal Innovations recently developed a high-speed system capable of streaming data capture at up to 1000fps in 100% parallel with fluid review. Using a 1280 × 1024 CMOS-based high-speed Camera Link Camera from Fastec Imaging (San Diego, CA; www.fastecimaging.com) interfaced to a Xcelera-CL PX4 PCI Express-based frame grabber from Teledyne DALSA (Waterloo, Ontario, Canada; www.teledynedalsa.com), the system is capable of capturing 1280 × 1024 images at speeds of up to 500fps or 1280 × 512 images at speeds of 1,000fps. Employing a RAID array of solid state disks, the Frame Glide software allows Terabytes of image data to be reviewed immediately, right up to the present moment, without interrupting the flow of incoming data.

The FG|SB system's ability to review image data in ongoing image capture have proved useful in medical applications, where, for example, physicians have used the display during gastroenterological procedures such as colonoscopies, where the physician requires an interface that allows medical students to review content without interrupting ongoing image capture.

Temporal Innovations has also added high-speed frame filtering and frame re-ordering to the system. This filtering and reordering can be based on image metrics such as color and scene complexity, or secondary time-synchronized sensor data such as temperature and wind speed.

In demonstration of a functional prototype, the software used RGB color information extracted from each frame of a 13 min 30fps video taken driving around the company's neighborhood, to locate blue cars in less than 1s. The query was a simple text string written in a scripting language designed for this purpose. A real-time demo of this example can be found on the company's website.

"Such functionality can find a particular subject within tens of hours of security video, or to compare thousands of repetitions of an automated assembly step looking for fluctuations, says Durlach."

"The FG|SB's ability to provide near-instant results stems in large part from decoupling the metric extraction (say color) from the user's queries", says Durlach. "For a large video dataset, extracting diverse metadata may take some time. But once that step is complete, the user may then ask complex queries and get answers immediately. Seeing images filtered and reordered in such a way can help rapidly isolate specific events."

However, it may be often the case that an operator does not know the specific metric (such as color) or frequency that will provide the required results. Here again, the software's ability to play "what if scenarios," could, for example, allow a systems developer to correlate factors such as temperature, sound, or humidity with captured images using sensors coupled to the PC. In such a scenario, the developer could test hypotheses such as whether specific sounds were related to specific imaging events, and automatically isolate images based on these external factors.

For complex, domain-specific areas, the FrameGlide environment can use third-party metrics. "In many fields, there are experts who have developed sophisticated metrics addressing their particular area of interest – whether that is identifying a particular person's face from within a crowd, recognizing an enemy fighter jet from its sound signature, or detecting a potentially cancerous tumor from an MRI scan", says Durlach.

For qualified parties, Temporal Innovations offers free evaluations of the FrameGlide|Spin Browser system.