3D imaging estimates pothole volume

Using a Microsoft Kinect mounted on a tripod, researchers at Nottingham Trent University are using 3D imaging to provide more accurate analysis of potholes.

As well as creating dangerous driving conditions, potholes may collect water that may further degrade the condition of the road. Preparation for pothole repair is as important as the repair itself since a fast and accurate measurement of the extent and severity of a pothole can help to identify the potential repair required. Potholes with depths greater than 40mm and length or width greater than 300mm may need to be repaired quickly. Accurately estimating the amount of material required to fill the potholes is also important since a lack of the correct amount of material will lead to inferior compaction while an excess quantity can lead to over banding at the edge and material wastage.

Manual surveying such potholes is slow and inaccurate since it is difficult to measure the volume of the pothole due to its irregular shape. To overcome this subjective approach, accelerometer-based measurement, stereovision, laser and LED based 3D profiling techniques have been used to perform this task. However, although vibration-based methods that use accelerometers are simple, they cannot not provide volumetric measurements but instead provide a relative measure of the degraded road condition. Limitations also exist with stereo vision and laser- based methods since stereovision cannot be used under low lighting conditions and, while laser based estimation methods provide accurate volumetric estimation, they are expensive and require specially equipped vehicles.

To overcome these limitations, Dr. Khurram Kamal at the National University of Sciences and Technology (Islamabad, Pakistan; www.nust.edu.pk), Dr. Senthan Mathavan at Nottingham Trent University (Nottingham, England; www.www.ntu.ac.uk) and Dr. Mujib Rahman at Brunel University (London, England; www.brunel.ac.uk) have developed a 3D system that uses Microsoft's Kinect to perform pothole metrology and volume estimation.

Kinect is equipped with an RGB camera, an IR sensor or camera, microphones, accelerometer, and a tilt motor for motion tracking. The IR camera of the Kinect sensor provides 640 x 480 pixel depth images and operates at 30 Hz. The resolution of these depth images can be increased further to 1280 x 1024 pixels at a reduced frame rate of 10 Hz, making the Kinect suitable for high resolution applications. Since the working range of Kinect is between 800mm and 4000mm, it can provide depth values of a given scene, directly in millimeter range. The pixel value of any given point in the scene, in conjunction with its corresponding depth value, allows the calculation of its 3D coordinates.

Images can be acquired from the Kinect sensor using commercial and open source software such as the OpenNI framework, an open source SDK used for the development of 3D sensing middleware libraries and applications (www.openni.org) and Open KINECT, a cross-platform API. For pothole analysis, depth images were acquired using the KINECT sensor using the Open KINECT software under Ubuntu Linux (www.ubuntu.com). Mounted on a tripod, the Kinect sensor captures an area of 1.12 m x 0.840 m per image (see page 15).

After images are captured, point-cloud meshes are generated using MATLAB from MathWorks (Natick, MA, USA; www.mathworks.com) for better visualization (top right). These 3D point cloud meshes allow the calculation of metrological features such as mean and maximum depth, volume, centroid, eccentricity and perimeter length. A numerical integration scheme in MATLAB has been employed on the 3D cloud data to calculate the volume of potholes. For the depth image of the pothole illustrated above, for example, the mean depth is 5.9 mm with a maximum depth of 31 mm. In addition, the volume estimate is 1, 760, 000 mm3 (i.e. 1.76 liter) and the perimeter 822 mm.

The cross-university team has benchmarked Kinect on artificial defects of known geometries, under different environmental conditions resembling actual pavements. The resulting volumetric accuracy of the sensor is around 5%. In future, they intend to benchmark the Kinect sensor for pothole metrology by imaging from a moving platform and study whether the device may be suitable to quantify other road defects in real-time.