Artificial Intelligence (AI) techniques such as Deep Learning and Computer vision have a wide variety of use cases these days, and the number of use-cases will only increase further in the coming times. One use-case which showcases the versatility of AI and computer vision in benefitting society at all levels is the creation of a smart city. A smart city is a city that utilise different types of sensors such as cameras to collect data, where this data would then be used to improve the quality of life of the citizens and improve operations across the city.
1. Traffic Monitoring
A smart city would be able to monitor and detect the movement of the vehicles throughout the city. This brings many benefits to the citizens, depending on application, in areas such as security and safety, reducing power consumption, helping citizens to be efficient with time and much more.
1.1 Accurate Vehicle Detection
Accurate vehicle detection is an essential building block towards effective traffic monitoring. Vehicle detection is where the detector can identify and produce the bounding area of vehicles, assigning unique IDs to each identified vehicle.
There are a few different ways to detect vehicles, from traditional and conventional machine learning approaches, such as Histogram of Oriented Gradients (HOG), R-CNN, Haar Cascade Classifiers, etc, to the modern deep learning engines. Deep learning algorithms revolutionise real-time object detection, providing state-of-the-art accuracy while still having a high inference speed, thus allowing it to achieve good detection accuracy when used in real world video applications. For example, YoloV7, developed in 2022, surpasses current state-of-the-art object detectors in both speed (having at least 30FPS) and accuracy (56.8% AP) when benchmarked using the COCO dataset using Nvidia V100 GPUs [https://arxiv.org/pdf/2207.02696.pdf].
1.2 Accurate License Plate Recognition
License plate recognition is where a detector can extract the license plate characters from an image containing vehicle license plate(s). When used together with vehicle detection, the movements of individual vehicles throughout the city can be monitored across different locations. As vehicle license plates are tied to the owners of said vehicles, accurate license plate recognition, when used to benefit the citizens of the smart city, could be used to dramatically reduce the crime rates involving vehicles in the smart city.
There are many crimes involving vehicles, for example robberies, ‘hit-and-runs’, drunk driving, reckless driving and more. When the city has monitoring systems in place for detecting these behaviours through computer vision and AI, using license plate recognition as a building block, perpetrators can be traced directly to their vehicles performing the crime, and they would receive warnings or be charged with the crime automatically. Thus, acting as a great security deterrence against such crimes.
1.3 Traffic Flow Detection
As vehicles are detected on roads, the direction of movement of vehicles can be automatically calculated through computer vision algorithms. Jisang Yoo et al. explain a method of detecting the direction of movement of detected objects in their article ‘Moving Object Detection Using an Object Motion Reflection Model of Motion Vectors’ [https://pdfs.semanticscholar.org/ced9/d8977c163e1d2d22af10586043d477efecfc.pdf].
The calculations required in this example involve road estimation, system motion estimation, and lastly moving object detection. Using traffic flow detection, the smart city would be able to detect the expected behaviours of citizens in the city and detect any erratic behaviours on the road by drivers such as drunk or reckless driving.
1.4 Traffic Congestion and Heat Map
Using vehicle detection, it would also be possible to automatically detect the areas which have the highest traffic heat throughout the day, as visualised in Figure 2. This information can then be used in the smart city to plan ways to reduce traffic heat in locations with high traffic heat, through the building of additional roads and highways or other methods. This would allow the citizens to be more efficient with their time during their commute throughout the day. Besides, traffic heat data can be available to the public, allowing the citizens to plan on their time in real-time, and avoid commuting during hours which have high traffic heat on the road.
2. People Detection
Besides detecting vehicles, a smart city which detects people will also bring many benefits to the citizens of the city.
2.1 Left-behind Object Detection
Firstly, objects which are left behind by people in the smart city would be able to be detected. These left behind objects could be small but important items such as wallets or credit cards, where upon successful detection, could be returned safely to identified owners. Left-behind object could pose a potentially serious security threat to the public, containing illegal and dangerous substance such as explosives or illegal drugs. Being able to identify such item and owner of said item can greatly increase the security of the city.
2.2 Detection of People Loitering or Trespassing
The system would be able to detect any individuals that are loitering within certain demarcated zones. This can greatly improve safety , by identifying anyone loitering in high risk area, for example construction site, or zone handling heavy machineries or cargo. It also contribute greatly to security by flagging any suspicious individuals loitering near government or private buildings for long durations.
By utilising different classes of detection system, the smart city can develop a robust security and safety system. One example is by combining people and smoke detection, security personnels will be able to use the hybrid system to detect and identify any rule breakers smoking in non-smoking zones. Another example is utilising people detection along with dangerous object detection such as knives to provide real-time information to authorities for efficient crime prevention.
The system can also be use to detect and identify any trespassers in areas designated as high security zones. The difference between loitering and trespassing individuals would be loitering people are those that gather at a location, while trespassing individuals are people that cross into designation areas. These scenarios and specifications will be configurable in the system in various ways. Some configurable matrix will be specifying regions of interest, threshold lines such that crossing of said bounding, or dwelling in an area for a set period of time will be flagged to authorities.
2.2 Crowd Monitoring and Heat Map
A crowd monitoring system can be set up in public spaces within the city, such as airports, city centres and during specific time periods such as festivals and celebrations to benefit its citizens.
The system can improve the safety of citizens by identifying potential security risks in locations which are overcrowded. The density of detected crowds within regions will be quantified and can be displayed within a heat map in the monitoring system. An example of a scenario involving overcrowding was the Seoul Halloween crowd crush in October 2022. The implementation of a crowd monitoring system on the streets of Seoul could potentially have aided authorities by providing real-time data on crowds in different locations in the city, allowing for better and more efficient crowd management.
