FPGA-based system enables real-time extended-focus imaging

Silicon Software (Mannheim, Germany; https://silicon.software) and Optotune AG (Dietikon, Switzerland; www.optotune.com) have devised an enhanced “Extended Depth of Focus” (EDoF) functionality using a programmable frame grabber and focus tunable lens for real-time operation.

From a continuous image acquisition sequence of up to 20 individual frames with varying focal length, a sharp final image with an expanded range of focus is generated in real-time. Comprising the focus tunable EL-10-30-Ci lens from Optotune and Silicon Software’s programmable microEnable 5 marathon VCX-QP frame grabber equipped with the EDoF application, the system has been highlighted by both companies at several recent events.

From a continuous image acquisition sequence of up to 20 (currently) individual frames with varying focal length, a sharp final image with an expanded range of focus is generated in real-time.

The system’s fast image acquisition allows 20 images to be acquired per sweep, and then in real-time, delivers the generated image together with a depth/index image as well as a contrast image which can be viewed in the YouTube video at: http://bit.ly/VSD-FOST.

To achieve this result, the EL-10-30-Ci mounts in front of a 12mm S-mount lens on a monochrome JAI (San Jose, CA, USA; www.jai.com) Spark SP-5000M-CXP2 camera. The camera’s 2-channel CoaXPress interface achieves a total bandwidth of 12.5 Gbit/sec at 5-megapixel resolution and a frame rate of 400 fps.

VisualApplets pixel-precise simulation combined with the system’s fast image acquisition allows 20 images to be acquired per sweep, and then in real-time, deliver the generated image together with a depth/index image as well as a contrast image.

The entire system is controlled by Silicon Software’s microEnable 5 marathon VCX-QP frame grabber with a graphically programmable FPGA, which in turn has a 4-channel CoaXPress interface as camera inputs, 2 GB on-board RAM storage as internal image processing storage, and DMA1800 technology, based on PCIe x 4 Gen2 for image transfer to the host-PC storage.

In the frame grabber’s FPGA, the image is calculated with the expanded focus range from the image stack it receives. The EDoF application, programmed with the VisualApplets graphical development environment, executes the image calculation and output as well as synchronous control of the camera and a TR-CL180 lens controller from Gardasoft Vision (Cambridge, UK; www.gardasoft.com ), which dynamically alters the lens focus within milliseconds.

Sharp EDoF Image Calculated

The EDoF application running on the frame grabber’s FPGA controls image acquisition consisting of 20 individual images each, set between 1 to 4 diopters, with an image frequency of 400 fps. The 20 images of varying sizes are first individually geometrically corrected, with minimal latency, along the x- and y axes using bilinear interpolation.

The application then calculates the local contrast for every pixel of the image using a high-pass filter. It calculates the point with the highest local contrast from the individual image via looping. The result is the level with the highest contrast and thus the sharpest representation. From all the sharp image points, the image processing system outputs two final images at the end: an index/depth image (depth map) with the index of the image in question with the highest local contrast and the completely sharp image.

During 50ms, the tunable lens covers a range of 4 diopters during which 20 images are captured. These are then scaled and combined to an extended depth image on the fly by Silicon Software’s microEnable 5 marathon VCX-QP imaging processor.

This high-speed, image-processing system using a focus tunable lens is especially well suited for inspection systems with high data throughput, since the frame grabber executes the calculation in a short period of time during image acquisition.

Preferred applications are PCB, semiconductor or general electronics inspection, during which the form of the object to be inspected must be calculated and reconstructed, as an example, for the testing of solder joints on circuit board components. Further applications include the inspection of deep objects or fluids, such as those in bottle production or in pharmaceuticals, where foreign particles must be identified at a 100% rate.

“The EDoF system with integrated, focus-tunable lens can process image stacks up to 50 individual images with deterministic latencies and high speed. The application running on the FPGA directs the lens controller, the camera and the image processing from acquisition to processing to image output”, explained Dr. Klaus-Henning Noffz, Managing Director of Silicon Software.