Machine Vision Software Detects Safety Risks at Seaport

Using AI-enabled vision software, a port authority in the United Kingdom has dramatically reduced the number of low-level safety incidents.

The Bristol Port Company (Bristol, UK), located 120 miles west of London and south of Wales, implemented SiteLens from RoboK (Cambridge, UK). The product uses deep learning models to analyze video streams from existing industrial CCTV surveillance cameras and provide insights on low-level safety issues, primarily those involving interactions between people and machines. The low-level safety issues, or "near misses," involve human behavior (i.e., walking too close to a forklift) or systemic situations (i.e., lack of physical barriers between warehouse employees and semitrucks).

“We were having too many low-level incidents, and we were not harnessing the potential of the existing CCTV installations to capture and correct these incidents,” explains David Brown, CEO of the Bristol Port Company, a privately owned British company that also has statutory authority for the harbor. Those “low-level” incidents included employees not wearing personal protective equipment, not using designated walkways, or forklifts coming to close to each other.

Hao Zheng, co-founder and CEO of RoboK, which has secured $4 million in funding, explains that safety “near misses” occur regularly at ports because there is so much daily activity involving interactions between people and moving vehicles. “They are very busy people working in these complex, dynamic environments,” she says.

By reducing the “near misses,” organizations such as Bristol Port Company also decrease the risk of serious accidents involving injury or death, she says. While other software products provide real-time alerts for high-risk behavior with potential for severe consequences, RoboK focuses on providing video clips and analysis of more frequent low-level incidents.

Tracking Behavior with Deep Learning

To implement the solution from RoboK, Bristol Port Company started in June 2022 with pick and pack processes for bottled beverages. This high-volume warehouse operation at the port handles more than 2 million pallets of the beverages annually.

RoboK accessed the video feed via a private network IP connection to video recording and management software, Avigilon Control Centre from Motorola Solutions (Chicago, IL, USA), on a central server at the port. The CCTV cameras are from a variety of vendors.

RoboK then processed the video streams in batches in the cloud, reviewing them for safety incidents using a pretrained neural network. Visualizations based on aggregated statistics—such as heat maps or graphs—also were produced with SiteLens, and managers at the port accessed the data via web-based dashboards, which are updated continually. They also reviewed short video clips of safety incidents, which are stored in the cloud.

Zheng notes that the port’s warehouse operations are well served with surveillance cameras because this is where merchandise is stored after it is unloaded from ships and before it is repacked for delivery to distribution centers or retail outlets. This meant that RoboK could tap into video feed from those cameras, which is a less costly approach than requiring the port to invest in new camera infrastructure.

The typical image resolution from these cameras is adequate for the software, Zheng explains. “I am not so worried about the resolution provided I have something there,” Zheng explains. The software also can work with video streams that contain varying perspectives if, for example, an employee monitoring the feed zooms in on a scene, she adds.

In addition to tracking the aggregated KPI information, Brown says the port has been using the short clips for education and training. “The supervisor can mentor, coach and educate the individual not to make the same mistake again. The whole culture is about behavioral safety to empower the individual to make the right safety choice,” Brown explains.

Dramatic Drop in Safety Hazards

The results from the single warehouse operation are compelling: the pick-and-place operation went from 90 potential hazards per day down to “low single digits,” Brown says. As a result of the success so far, Brown plans to expand use of the software to operations throughout the port, which has a total of 160 CCTV surveillance cameras in use.