Stereo vision makes construction vehicles safer
Owners and operators of large construction and mining vehicles must ensure that their machines work effectively without injuring on-site personnel. Because the machines are often very large, it may be difficult for an operator to visualize how specific aspects of the machine are performing or whether on-site personnel have been placed in danger.
A number of companies are now employing machine-vision systems to both prevent unsafe operation of the equipment while at the same time ensuring workers’ safety.
Ten years ago, for example, Motion Metrics (Vancouver, BC, Canada; www.motionmetrics.com) in collaboration with Alberta Research Council (Edmonton, AB, Canada) developed an embedded, intelligent, single-camera machine-vision system to detect missing bucket teeth on large excavators (see “Vision system detects missing shovel teeth,” Vision Systems Design, October 2001).
Since the teeth are manufactured from hardened steel, lost or broken teeth can cause serious damage to associated machinery in the downstream processing plant. Real-time monitoring of the bucket is then used to dynamically warn the operator of any tooth loss and thus ensure that no downstream damage can occur.
Now, designers at Arcure (Orsay, France; www.arcure.net) have taken this concept one step further, developing a 3-D system known as Blaxtair that can detect workers that may be in unsafe range of heavy machinery (see figure). Designed to be mounted on the rear or sides of large construction vehicles, the system can automatically alert an operator to any human shape in close proximity to the machine.
Rather than use time-of-flight or structured-light methods to perform the 3-D imaging, Arcure has chosen instead to take a more conventional approach, developing the Blaxtair unit with dual Blue Cougar X GigE Vision cameras from Matrix Vision (Oppenweiler, Germany; www.matrix-vision.com).
“Because it was necessary to mount the Blaxtair camera unit at some distance from the control unit located in the cabin,” says Patrick Mansuy, president of Arcure, “the system required the flexible, low-cost interface provided by the GigE interface of the Blue Cougar cameras.”
Perhaps more important, operating the camera systems in outdoor environments demands automatic control of both automatic gain and auto-exposure and a system in which these parameters can be synchronized between two cameras. “This was possible since both the automatic gain and auto-exposure of the HDR WVGA CMOS imagers from Aptina [San Jose, CA, USA; www.aptina.com] used in each of the Blue Cougar X cameras can be dynamically adjusted and synchronized over the GigE interface,” says Mansuy. By fitting these cameras with a short-focal-length S-mount lens, a large stereo field of view can be obtained.
Like most dual-camera stereo systems, the Blaxtair system uses the principle of epipolar geometry to generate 3-D images from the two calibrated stereo image pairs. After the two images are transmitted to a host PC in the cabin of the construction vehicle, epipolar geometry is used to recreate a 3-D scene.
For the system to understand the shape and orientation of objects within this scene, Arcure licensed a patented algorithm originally developed by researchers at the French Alternative Energies and Atomic Energy Commission (CEA; Gif-sur-yvette, France; www.cea.fr).
In operation, this algorithm can be programmed to scan the 3-D data and determine whether a human shape is in a specified range of the 3-D camera. If an obstacle is detected, the system can then flag a warning to the operator within 300 ms of the occurrence.
According to Mansuy, the system is currently being field tested by two very large French construction companies on different types of construction vehicles.