HTX engineers are using video analytics and AI to help the Home Team save lives more effectively and swiftly at sea.
The S&S CoE engineers testing the system. (Photo: HTX)
The waters around Singapore bustled with activity as vessels including tankers, tugboats, and pleasure craft navigated through the stunning seascape.
A team of engineers from HTX’s Sense-making and Surveillance (S&S) Centre of Expertise (CoE), was onboard a Singapore Civil Defence Force (SCDF) Marine Rescue Vessel (MRV) in the waters close to Sentosa. The view of the outlying islands before them was spectacular, but the team’s focus was on a tiny white dot – barely visible – on a screen mounted inside the vessel.
A trial using video analytics and artificial intelligence (AI) to identify humans in the water was underway and the white dot on the screen was actually a life-sized mannequin floating far out at sea. The trial was done in collaboration with SCDF Marine Division, which responds to emergency incidents at sea.
Using marine video analytics and AI to save lives
Currently, SCDF adopts a manual sighting and search approach using binoculars in their search and rescue operations.
“As search operations may be prolonged, we are developing an in-house video analytics algorithm to improve lifesaving capabilities as well as to provide automatic and reliable detection of objects of interest in real-time,” explained Priscilla Koh, Lead Engineer (Tech Management), S&S CoE.
As images retrieved at sea are often of poor quality (e.g. due to the poor lighting conditions, moving vessel), the project team has adopted several image enhancement techniques, including colour and contrast enhancement to improve the images’ quality.
These enhancements are also applied to video footage, along with a fusion of multiple AI models including Deep Learning Detection and Colour Detection. Colour detection and enhancement techniques are especially useful for marine rescue since specific colours such as red and yellow are generally unnatural and distinct colours out at sea.
The S&S CoE team hopes to deliver a deployable prototype solution with map-based Graphical User Interface (GUI) by the end of the proof-of-concept phase of the project.
The map-based GUI optimises user experience. (Photo: HTX)
Overcoming challenges in marine video analytics
Despite the advancements in video analytics, the use of intelligent video analytics in the maritime sector is nascent, according to Benjamin Cham, Lead Engineer (AI Engineering), S&S CoE.
“This is largely a result of the challenges and unpredictable conditions at sea, including sudden changes in illumination and perspective as well as distractions of all kinds such as debris and clouds at sea,” he explained. “We are still early in this journey.”
While video analytics can be a powerful tool for emergency responders in the maritime context to identify humans on or under water, the marine terrain is especially challenging as varying sea conditions play a big part in the detection and accuracy of the results. In addition, minimal visibility at night makes detection especially difficult.
The engineers also had to overcome the lack of quality data for the sea domain as the field is relatively niche, with very limited references available. Due to the lack of open-source data, the team has been generating training data for the AI model from synthetic infrared thermal imagery (see image 1 below). In addition, the limited amount of sonar data and the paucity of ongoing research in this specialised area led the team to superimpose clips of human bodies on sonar images to generate training data for the AI model (see image 2 below).
Image 1
Image 2
(Images: HTX)
“This is an in-house development curated to SCDF’s needs to enhance their operational capabilities in search and rescue operations,” Priscilla said. “By providing improved situational awareness and faster response times, our system can potentially help save lives and improve the chances of a successful rescue.”
“We are very excited about the potential of this technology and what it can do to help enhance the Home Team’s capabilities,” she added.
This project was recently showcased at the Milipol Asia-Pacific – TechX Summit exhibition held at the Marina Bay Sands convention centre from 3-5 April 2024.