How Smart Cameras Are Used in Digital Twin Platforms

Andrea Sorri

Ever since the rise of popular games such as “The Sims” and “Second Life”, virtual worlds have received attention, and the potential new “metaverse” has reinvigorated consumer imaginations about the potential for virtual worlds. While far removed from gaming, the use of virtual modeling can be beneficial in helping smart cities achieve their objectives more quickly.

Processes and journeys across the city limits can be mapped in virtual models—exact replicas of city environments, so-called “digital twins”—which can aid in understanding the city’s dynamics and rhythms. This can be done retrospectively and in real time. Using this information, insights on different factors, such as congestion rates, response times, and air quality can inform future planning decisions.

Although the possibilities of virtual worlds are still being explored, these activities are already in progress in different sectors. For smart cities, data from sensors can be collected from different areas and collated into digital twin platforms to help officials understand and predict the impact of different scenarios. These may include reviewing the effects of road closures on traffic flow, working out the best evacuation routes if an incident occurs, or measuring the impact of activities to improve air quality or energy efficiency.

So, where do smart cameras play a role? Network surveillance cameras equipped with the right software act as critical sensors to capture visual data from around cities and record events of interest.

Digital Twins Are Built on Data

Many industries, such as manufacturing and logistics, use digital twin platforms to view and map assets. However, before discussing how digital twins are used, it’s important to know what they are.

Digital twins are virtual representations that serve as the real-time digital counterpart of a physical object or process. In a smart city context, these are virtual models of the city built using data collected from Internet of Things (IoT) sensors placed at strategic points. Understandably, data from a range of sources is needed to build an accurate model of a complex structure such as a city; it can vary widely from air pollution levels, noise levels, weather conditions, and the movement of vehicles, bikes, and pedestrians through certain areas of the city.

Aside from being innovative, the digital twin platform becomes very useful when city officials attempt to understand movements, behaviors, and the subsequent impact of different events. The platform measures metrics across certain periods of time to understand single events or longer-term impacts.

For example, if a festival takes place in a particular city square every year, data can be gathered via network surveillance cameras to find out how people are accessing the event—which entrances tend to be most crowded, how people are arriving at the venue (for example, on foot or via public transport), and peak times for traffic. Using this information, city officials can extract learnings and make assumptions about the behavior of crowds at future events there. This can also inform incident response planning, as information about the least crowded entrances can be used to plan potential emergency response routes.

When monitoring the festival as it takes place, it will also be possible to compare people flow with the predicted model mapped across the digital twin. For example, operators might notice that the real-time data shows one exit is not being used as predicted during the festival. On closer inspection, they might find a blocker, such as an ambulance parked too close to the exit, and be able to make a call to move that obstruction and balance out the flow of people.

Planning for the Future by Understanding the Present

As mentioned, the insights gathered from the platform aren’t necessarily limited to a single event. By extrapolating the monitoring capabilities, the general movement of traffic around the city can be traced and mapped. This includes information about the number and type of vehicles present (for example, how many cars, trucks, and bicycles are operating in certain areas) and how citizens use the roadways over a period of time.

This information is invaluable for future city planning, as it can be used to inform decisions about proposed changes to urban infrastructure. As an example, Okazaki, Japan, has employed surveillance cameras to understand changes in people flows as part of its move to create a walkable city center. A total of 21 cameras—a combination of AXIS Q1615 Mk II Network Cameras, AXIS Q3518 fixed-dome cameras, and AXIS P5655-E PTZ cameras—were installed around the train station, in the shopping district, along the pedestrian paths, and on the footbridge to obtain data on people counts at crucial locations within the city center. The data acquired and analyzed are currently shared with stakeholders including City Hall, city event planners, developers, and private-sector businesses considering a retail presence in the area. However, future applications beyond people counting are under consideration, with the aim of expanding this to other smart city capabilities including crime monitoring and prevention.

As data are collected from multiple sources and funneled into the digital twin platform, advanced analytics such as multidimensional AI can be enabled. This is where AI is applied to analyze different datasets to draw greater insights that can be applied at a macro level.

These learnings can be used when going through “what-if” scenarios, as the impact and outcome of an action can be reviewed. In this way, challenges can be anticipated and planned for in advance.

Surveillance Cameras Add Value by Providing Real-Time Visual Data

Among the different types of data collected into the platform, visual data are especially useful. The strength of digital twin platforms depends on the quality and the granularity of data input from sensors. These will feed information into the models, enabling the software to build accurate virtual representations. Network cameras are critical to this process, as they not only provide surveillance—capturing high-quality images that are the foundation for analytics insights—but also real-time visual feedback.

Importantly, with many network cameras already in position in crucial locations, it’s possible to take advantage of an existing install base for data collection, as well as deploying new cameras where required. For example, the metropolitan city of Milan in Italy carried out an integrated project to improve road safety, and this required installing additional network surveillance to collect the necessary data input.

A video surveillance system with intelligent image analytics, such as the ability to detect vehicles, people, and suspect behaviors, was adopted in lay-bys. Crossings were equipped with 360° cameras and photocells. Detectors were installed to record speeding cases instantly through a camera with a noninvasive laser sensor. These sensors were all connected through an integrated system, ensuring that data were collected, analyzed, and used to transmit alert notifications so operators were quickly made aware of a criminal offence or accident. As a result, main roads in Milan are constantly monitored, gathering real-time, high-quality information that enhances road safety.

Whether cameras are focused on main roads, key buildings or event locations, the video captured by these image sensors can be analyzed by applications at the edge to provide information related to particular scenes. While the latest network surveillance technology enables initial image processing and metadata extraction to occur through edge analytics, more complex image or metadata processing may take place server side or in the cloud. This information can then be funneled into a digital twin platform. Here, the data is typically represented as a 3D visual such as a city heatmap.

Cybersecurity Remains a Challenge

The richness of the detail exhibited in the digital twin model is directly proportionate to the quality and amount of data input. Achieving and maintaining this level of data input requires information to be transmitted from a number of sources, which ultimately increases the cybersecurity attack surface area. Unfortunately, smart cities are targets for attacks from malicious parties, and the scale of disruption caused by a successful breach can be significant. As a result, it’s critical for city officials to bear in mind the security of devices implemented address existing vulnerabilities and review the network regularly to ensure that the entire ecosystem is secure.

Supporting Smart City Goals—Now and in the Future

Technology implemented in smart cities will be used to improve efficiencies, streamline processes, and improve certain metrics related to smart city goals. These can be varied and range from reducing response times to decreasing crime rates and improving the availability of parking. Data collected in smart cities will ultimately be used to support these different goals, not only to improve liveability for residents but also to contribute to wider sustainability objectives.

Digital twin platforms play a role by helping city officials better understand processes, take specific actions, measure progress, and predict outcomes. For example, if improving air quality is an objective, city officials can review the digital twin model to see where and why congestion occurs. They can then make specific plans to proactively address the cause, thereby alleviating the problem. One thing that must be borne in mind is the security of the platform to ensure the insights are trustworthy and have not been tampered with.

Network surveillance cameras will contribute to achieving these objectives by providing visual data to help monitor activity in real time and inform planning for future events or changes. The verification of events also helps build digital trust in the technology, leading to increased adoption of digital twin platforms. Ultimately, these types of virtual models will become a critical factor in moving smart cities closer to their objectives and realizing the associated benefits for residents and visitors alike.