On December 1, 2019, the Australian state of New South Wales (NSW) rolled out a network of artificial intelligence (AI) cameras capable of capturing and drivers using a mobile phone while driving.Arnaud Destruels
In 2018, 354 people died on NSW roads, and (as of December 1, 2019) the figures were on track to match 2018, with 329. According to NSW government figures, on average, 23 casualty crashes involving 32 people, happen each year involving a driver using a handheld mobile phone. However, “the contribution of mobile phone distraction to road trauma is underreported due to difficulties with obtaining conclusive evidence at crash scenes.”
Of course, this is a worldwide issue, and World Health Organization data state the use of mobile phones while driving brings a four-fold increase in the risk of being involved in a crash.
Additionally, a 2018 literature review of 4,907 articles, published in the Journal of Emergencies, Trauma, and Shock, found that quoted statistics varied widely, but cited one (outlying) paper that attributed 44.7% of road traffic injuries and fatalities in the sample studied were caused by mobile-phone-based distraction. The authors also believed these studies (on average) “underestimate the actual prevalence of road traffic injuries related to mobile device use.”
Indeed, more than 30 countries around the world have made it illegal to use a handheld device while driving - some with significant fines, and even jail sentences: Oman, for example, has a maximum punishment for mobile device use while driving of up to 10 days in jail and a fine of 300 OMR (Approx. $780 USD).
But the risk of getting caught in many places isn’t high, with it being costly to police and hard to automate. In particular, glare prevents cameras from seeing into the car, meaning drivers are often willing to take a risk. 2018 survey data from the UK states that, despite the known consequences, 25% of drivers admit to doing it, while Australian research suggests one in six people regularly send texts while driving. A US study also shows that 27% of American adults admit to having read or sent texts while driving.
The New South Wales roll out is part of an initiative to cut fatalities and injuries by 30% by 2021 and follows a 6-month trial that captured 100,000 drivers using their phone illegally.
The technology being used
According to New South Wales’s press release, the system uses multiple cameras, and its accompanying video / press images suggest those capturing mobile use are positioned on a gangway above the vehicle and output images in black and white. It uses an infrared flash to “capture clear images of passing vehicles in all traffic and weather conditions”.
And while not shown in the video / press images (likely for data protection reasons), additional intelligent transportation system (ITS) cameras will be in use to capture the license plate and driver.
And while the New South Wales government press release says the overhead images control for weather and the varying nature ambient light, these front-facing cameras will not, making it harder to capture the face of the driver.
Resolution will likely be HD or higher and will likely transmit over GigE, both because it’s the most common standard used in ITS systems for many countries, but also because it allows for accurate synchronization between the gangway-mounted cameras and the front-facing ones using the IEEE1588 precision timing protocol; indeed some cameras - such as from Sony - can now act as the master device in a PTP system.
From these images, artificial intelligence software is used to determine those most likely to display illegal mobile phone use before being passed to a human for verification.
A mobile version of the technology is also set to be deployed.
Will it work?
In this case New South Wales has set a fine of AUD344 (€212, $235), but more interestingly increased it by a third (AUD457 - €281 / $311) when in a school zone.
Cameras coupled with significant fines and a fear of being caught have been proven to act as a very cost-effective deterrent. Data from California Polytechnic State University showed red light violations dropped by c. 60% when a camera was present.
But, one of the key limitations in such systems is the ability to prove who’s driving.
Technical capabilities of cameras have improved significantly in recent years. For example, automatic number-plate recognition (ANPR) cameras can automatically capture the number plate, the color, the make, and the type of vehicle, and then cross reference this against vehicle and criminal database. Glare, however, presents a significant challenge in proving who was driving and what key details about what’s happening in the vehicle, therefore limiting the effectiveness of the law.
As a conference paper presented by a Dutch team at 2013’s Beijing’s Four Continents Conference on road safety put it:
“[D]ue to e.g. lighting conditions, glare, coatings on car windows or intentional obstruction, the recognisability of the face of the driver on an evidence photo may be difficult. This reduces the prosecutability rate of the registered violations. Processing rejections if the owner was not driving also adds to the operational workload.”
The paper also highlighted that analyzing the photo is both labor intensive and costly, meaning authorities would spend more to police it than they could make in fines.
This can often be difficult and while data on prosecution rates is rarely published, data from the German state of Baden Württemberg reports a third of driving violations captured by its automated camera systems were stalled due to an inability to identify the driver, often due to glare.
For the new cameras, New South Wales has reversed the onus of proof from the state to the driver, meaning the ANPR system, coupled with the data from the overhead cameras should be enough to prosecute.
But there are fears that local courts would be overwhelmed. The government expects to capture 135 million vehicles per year and anticipates approximately 2.4 million of these will capture a mobile phone being used. A committee investigating potential effects of a roll out for Transport for New South Wales (TfNSW), however, was worried about this, with just 3% of cases being appealed, c.73,000 cases would end up in the courts, with a spokesperson for the committee saying:
“A simple review of these numbers indicates that this program runs the risk of overwhelming the local court.”
Should the onus of proof be on the government or the vehicle owner, I would personally suggest it’s fairer for it to be with the government. But this changes from country to country and glare creates a significant challenge to authorities seeking to reliably see beyond the windscreen.
Mobile phone use is not the only law that has proved difficult to police by cameras
Other rules affected by glare include the policing of multiple-occupancy lanes, as well as seatbelt and child-seat laws—and again, enforcing these enables countless young to be saved.
And, more recently, laws in several Australian states, Bahrain, Canada (all states but Quebec), Cyprus, England, Wales, France, New Zealand, South Africa, the UAE, and a small handful of US states have been enacted to prevent people smoking when children are in the vehicle. Finland, Greece, the Netherlands and Taiwan are set to join these, and similar laws are being debated in Germany.
Glare prevents the ability for cameras to see through the windshield in many conditions. Tellingly in the UK, where the smoking-in-cars-with-minors law has been in place for four years, there is one concerning statistic: just one fine has been issued for breaking this law. This may mean the law has worked perfectly and nobody smokes anymore; but it is more likely it is proving too much effort for police to act on and it has not been cost effective to automate it using camera technologies due to effects such as glare preventing a high-enough prosecution rate.
Technology is finally catching up
Last year saw the introduction of a new class of imaging sensor, one that integrates a four-directional (0°, 45°, 90°, and 135°) wire-grid polarization filter below the on-chip lens, each pixel can be assigned an angle of light, ordered in a 2x2 calculation unit. The sensor outputs 5.07 MPixel images (creating 4 x 1.27 MPixel images in each plane) at 23 fps. And by using the intensity of each plane it’s possible calculate the direction light and the degree of polarization and compile the images to create a clear image that can be used to increase the usability of the images regardless of the weather and light conditions.
Among the first industrial camera modules to use this sensor for ITS applications is from Sony Europe’s Image Sensing Solutions division, the XCG-CP510. Sony has also launched a software development kit to speed development on this new kind of sensor, with an example application for windshield-glare elimination in ITS applications.
This cuts the development time from c. 6-24 months (depending on the application and development team) to 6-12 weeks.
Conclusion
Some have argued that laws such as for mobile phone use (or for seatbelt use / smoking) had not been cost effective to enforce and therefore not fit for purpose. I’d argue they were just ahead of their time, technologically speaking.
Assuming Baden Württemberg’s data is indicative of the bigger picture—and there is no reason to expect it is wildly abnormal—then authorities are both losing out on significant revenues through lost fines and dangerous, illegal driving practices are going unchecked.
It’s been well documented that cameras and the increased fear of being caught have been shown to act as a strong deterrent to speeding and jumping red lights. And as local authorities adopt (and as the public become aware of) these new polarized camera module technologies, which are capable of seeing through the windshield reliably, it’s safe to assume that drivers will adapt their behavior as they did around red-light jumping, with less mobile phone usage, greater seatbelt use and reduced smoking in cars, all of which can only be a good thing.