ShadowCam gives autonomous cars the ability to see around corners

Accident avoidance is one of the top concerns facing developers of autonomous vehicles. If full autonomy—or class 5 autonomy, liter­ally having no human being at the wheel— is ever to become a reality, autonomous vehi­cles must be as entirely safe as possible. This means taking as many steps as possible against scenarios that could potentially lead to colli­sion. Being able to see around corners would give an autonomous vehicle a large advantage toward achieving this end.

ShadowCam, a new technology developed by researchers at the Massachusetts Institute of Technology (MIT; Cambridge, MA, USA; www.mit.edu), working in collaboration with researchers from the Toyota Research Insti­tute (Cambridge, MA, USA: www.tri.global) and with support from Amazon Web Services (Seattle, WA, USA; aws.amazon.com) dem­onstrates the potential to give advanced driver assis­tance systems (ADAS) the ability to read changes in il­lumination on the ground caused by dynamic ob­stacles—moving vehi­cles—that are not in direct line of sight.

The technology uses a region of interest tech­nique, focusing the camera on the ground ahead of the vehicle, in the area where the appearance of a shadow cast by an approaching object is to be expected. The Shad­owCam classifier pipe­line uses a pre-processing routine to enhance images with weak signal, i.e. shadows. The al­gorithm then analyzes the images, compar­ing a pixel-based metric calculated from se­quences of images against a safety threshold. If this metric is less than the safety threshold, i.e. if there are shadows, the vehicle is com­manded by the algorithm to stop.

Two experiments were conducted, the first of which used an autonomous wheelchair moving down a hallway equipped with one of two different cameras, in different experi­ments. The first camera was a Canon (Mel­ville, NY, USA; www.usa.canon.com) EOS 70D single-lens reflex (SLR) with EFS 17-59 mm lens, which was used in experiments where the position of the wheelchair and the shape of the hallway were denoted with AprilTag fiducial markers.

The second camera was a uEye UI-3241LE-M-GL monochrome, global shutter CMOS camera from IDS Imaging Development Sys­tems (Obersulm, Germany; www.ids-imag­ing.com), which was employed when a direct sparse odometry method was used to deter­mine the wheelchair’s environment and rela­tive position.

Both cameras were tested against beyond-line-of-sight objects that were moving (dy­namic) or stationary (static). The mean classi­fication accuracy—successful classification of an object (dynamic or standing) beyond line of sight—was around 70%.

Experiments were also conducted using an autonomous Toyota (Toyota City, Aichi Pre­fecture, Japan; global.toyota/en) Prius driving in a garage. The illumination in the garage provided close to nighttime driving conditions in terms of the amount of illumination. The lights of the autonomous vehicle were kept off, in order not to flood the ROI in front of the vehicle with light. The Toyota Prius was equipped with the uEye UI-3241LE-M-GL camera, and the ROI and ground plane were also annotated for the ShadowCam algorithm.

The algorithm was able to detect the ap­proach of a moving vehicle around the corner faster than a SICK (Düsseldorf, Germany; www.sick.com) LMS151 LiDAR sensor that was also deployed on the test vehicle.