Vision enhances undercarriage security

A vision-based inspection system automatically detects any modifications to the undercarriages of rail cars.

Christopher A. Millar,CEO, Gatekeeper, Sterling VA, USA

To ensure that drugs, weapons or even individuals are not illegally entering a country across its borders, customs and border protection officers must routinely examine the undercarriage of the cars of trains where such objects might be unlawfully hidden.

To help them with the task, engineers at Gatekeeper (Sterling, VA, USA;www.gatekeepersecurity.com) have designed a vision-based inspection system that automatically determines whether any modifications or additions have been made to them, alleviating the need for security checks to be performed manually.

As a train passes through the inspection system at speeds of up 20km per hour, a dual-view scanner embedded between the rail tracks captures video frames of the undercarriage of its cars from two separate views and streams these to image capture software running on a PC. By reading data from embedded AIE (RFID) tags mounted on the rail cars, the system also identifies and determines the orientation of the carriages.

The video frames streamed to the PC are processed and stitched to create two individual digital "fingerprints" of each undercarriage, and tagged according to their identification and orientation. The tagged fingerprints are then compared to a set of previously scanned anomaly free images of the same or similar undercarriages in a database.

Any differences between the two are automatically detected by the system software and highlighted on a display by a red ring which is placed around each one together with an image of the exterior of the suspect rail car which is captured using a separate external surveillance camera (Figure 1).

Figure 1: The Gatekeeper system captures images of the undercarriages of rail cars together with data associated with their orientation. The images are compared to a database of reference images in a PC and any differences between the two are highlighted.

Travel and orientation

The IP68-rated scanner employs a proprietary mirror arrangement that sits in front of a GigE Basler (Ahrensburg, Germany;www.baslerweb.com) VGA (640 x 480) monochrome machine vision camera. The optical system enables the single imager in the camera to simultaneously capture two regions of interest of the undercarriage at two distinct angles - a 60° forward and a 63° backward facing view. These angles allow the scanner to "see" over axels, crossbeams and/or in pockets under a carriage.

In all, there are a possible four sets of two images each that the system could potentially capture of the undercarriage of just a single type of rail car. This is because the two images captured by the scanner represent unique views of each undercarriage that are dependent on both the rail car's direction of travel and its orientation. Two images of an undercarriage of a rail car moving North, for example, will be distinctly different from those images captured of the same rail car moving South. Added to these possibilities is one that a carriage may be turned on a rail turntable resulting in it facing the opposite direction and then moving in either a northerly or southerly direction..

So for automatic inspection to be effective, the scanned images must be compared with reference images with the same direction and orientation.

Hence before making a comparison of the two scanned images of the undercarriage of a car with "safe" rail car images stored in its database, the system must establish the direction of travel of the rail cars and their orientation.

To determine the direction of travel, the system is triggered by one of two laser line of sight sensors placed ten feet apart along the railroad track. Once one of the sensors is triggered as the train moves past it, it provides the system with an indication of the direction of travel of the train as well as the location of the front of the train. The second laser line of sight trigger is then disabled.

To enable the system to ascertain the orientation of the rail cars, the system uses Automatic Equipment Identification (AEI) tag readers located on the side of the track. These read data from the AEI tags mounted on the rail cars. By determining where the tags are located on a car, the system can determine its orientation. In addition, each tag is encoded with data that helps identify the carriage together with other relevant information.

Triggering the system

Once the first laser sensor has been triggered as the front of the train passes it, the scanner starts the process of acquiring video frames of the rail car undercarriages. The frames are then streamed to an Intel quad Core 2.8 GHz PC through a Gigabit Ethernet switch at a frame rate of between 150 to 200 fps.

During the acquisition process, the undercarriages of the rail cars are illuminated with 625nm red LED lights mounted on six light rails manufactured by Raytech Industries (Middletown, CT, USA;www.raytechultraviolet.com) and assembled at Gatekeeper.

Two shorter light rails on the external side of each track are angled up and toward the viewing angles of the scanner and two longer light rails between the rails are adjusted to maximize illumination of the undercarriage (Figure 2). A 620 BP band pass filter from Midwest Optical Systems (Palatine, IL, USA;www.midopt.com) placed in front of the camera sensor ensures that only the light reflected from the undercarriage of the train from the banks of LEDs is captured.

Figure 2: The undercarriages of the rail cars are illuminated with six 625 nm light rails. Two shorter light rails on the external side of each track are angled up and toward the viewing angles of the scanner and two longer light rails between the rails are adjusted to maximize illumination of the undercarriage.

Aside from the photoelectric sensors used to determine the direction of travel and the front of the train, a third vertically mounted distance laser sensor is used to detect the beginning and the end of each of the following rail cars.

Once this sensor is triggered at the end of each car as the path of the laser reaches the track surface for a short period, the image acquisition process is halted. The process of acquiring video frames of the undercarriage of the consecutive rail car resumes once the leading edge of that car triggers the laser sensor again.

As the two video streams are received by the PC, they are stitched together using a proprietary overlapping frame stitching algorithm developed by Gatekeeper. The process of stitching the two sets of low resolution images enables the system to quickly produce two higher resolution composite images of the undercarriage of each rail car, rather than acquiring images from a larger area imager at a higher resolution which would take a prohibitive amount of time (Figure 3).

Figure 3: The process of stitching the two sets of low resolution images enables the system to quickly produce two higher resolution composite images of the undercarriage of each rail car. Once stitched, the system releases the two images of the car to a software comparison engine running on another PC.

Once stitched by the image stitching software module, the system releases the two resulting composite images to another PC. Having previously determined the direction of travel of a rail car and its orientation, the system software tags the two images of the undercarriage accordingly. The images are then compared with anomaly free undercarriage images stored in a database which are also tagged according to their type, direction of travel and orientation.

After each rail car has cleared the scanner, it takes between two and three seconds for the system software to compare the newly scanned rail car images to the images in the database. The two independent views provide the necessary amount of visual information to enable the comparison software in the system to detect objects on top of an axel or a cross beam, under a vehicle.

The system identifies any foreign objects or modifications to the undercarriage by automatically circling the images with a red ring and activating an audio and/or a visual alarm. An operator can then zoom in to the highlighted area on the display (Figure 4). As the operator zooms into the circled area, a reference image displayed on screen automatically does likewise, enabling the operator to look more closely at the changes or differences the system has identified.

Figure 4: The system identifies any foreign objects or modifications to the undercarriage by circling them with a red ring and activating an audio and/or a visual alarm. The operator can then zoom in to the highlighted area to look more closely at the changes or differences the system has identified.

Ease of inspection

Since the images of the undercarriages themselves would not provide any indication of the exterior appearance of a specific rail car, the Gatekeeper system also employs a 1Mpixel TCP/IP based video surveillance camera from Axis Communications (Lund, Sweden;www.axis.com) which captures images of the front of the rail cars.

Triggered by the same photoelectric sensor which indicates the start of each carriage, the scene camera is mounted by the side of the railroad tracks where it captures an image of the leading edge of each of the cars of the train. The image is then transferred to the PC over a TCP/IP interface where it is stored on a database together with the associated images of the underside of each car.

Once the system has identified a suspect rail car undercarriage, it can then present an image of the exterior of the car on a display, enabling officials to quickly identify which of the cars on the train they must inspect through its characteristic markings or color. Should the system be unable to identify a specific railroad car undercarriage, it will highlight the unknown images of the undercarriage to the operator who will then be able to add and tag the new images to the system database or reject the images.

To date, several Gatekeeper systems have been installed at border crossings in the US and at nuclear plants overseas. Several such systems have been deployed to discover where changes have been made to rail undercarriages at the US/Mexican border to detect contraband goods and illegal immigrants, while others in Russia monitor the undercarriages of trains carrying goods into and out of nuclear plants.