[FEATURED NEWS] Taking SCDF training to the next level with XR and VR

HTX and the Singapore Civil Defence Force (SCDF) partnered up with academic and industry collaborators to create more immersive simulation training for SCDF officers using Extended Reality (XR) and Virtual Reality (VR). Here’s what these new technologies have to offer.

SCDF officers in action during a VR training session held at the Civil Defence Academy (CDA). (Photo: HTX)

The boundaries of our physical reality have long been pushed by Virtual Reality (VR) and Augmented Reality (AR). What’s the difference? VR immerses a user in a fully virtual environment while AR overlays digital content on top of the real world (think Snapchat filters). Then there’s Mixed Reality (MR) which, as the name suggests, integrates VR and AR.

Extended Reality (XR), on the other hand, is the newest kid on the block. It comprises all three – VR, AR, and MR – and combines the physical and virtual spaces together.

While XR is yet to be available commercially, we’re using this latest tech in the Home Team’s homeland security efforts by incorporating it into our simulation training to level it up.

Working with the National University of Singapore (NUS) CUTE (Connective Ubiquitous Technology for Embodiments) Center under the NUS Smart Systems Institute, HTX and SCDF developed an XR system with immersive and interactive elements to train SCDF officers in managing Road Traffic Accidents (RTA) and Hazardous Materials (HazMat) incidents.

Suit up to engage your senses

The XR system features a Multi-Sensory Suit (MAV Suit) which provides haptic feedback to officers and gives off smells, such as smoke, that you’d encounter in a live-fire scenario. The suit’s vest also emits heat so the officer wearing it would feel as if he were at an actual fire ground. This innovative system is the first of its kind because these haptic elements were not available in previous simulation suits.

As part of the training, SCDF officers wear VR headsets and MAV suits to immerse themselves in the simulation and use remote controllers to navigate the virtual world. The SCDF officer to the left is using a remote controller to put out a fire in the VR simulation. (Photo: HTX)

Realistic mock-ups

In a real-life RTA scenario, firefighters will first put out any fires before ensuring that the vehicle is stabilised so that they can safely operate on it without it moving. They will then attempt to remove the vehicle door with a spreader cutter so that they can extricate the unconscious casualty trapped inside.

The XR set-up for RTA simulation training. (Photo: HTX)

This procedure is recreated in the virtual world. In the XR training, trainees interact with a physical car frame mock-up by using a mock spreader cutter to operate on it. Both these mock-ups have built-in sensors and magnets that provide haptic sensations (such as vibrations) to replicate the experience of using a real spreader cutter to cut through a vehicle’s metal.

An instructor monitors the trainee’s performance from a computer, while another officer is on standby to assist. (Photo: HTX)

As fumes are released into the air in real-life scenarios, the MAV Suit is made to reproduce similar scents of the chemical substances that are safe for trainees to perform HazMat identification without exposure to the harmful effects of the substances.

The POV of a trainee watching his teammate as the latter works to stabilise the vehicle after the fire has been put out. (Photo: HTX)

This system is a step up from the usual RTA and HazMat training methods where designated wreckage sites with scrap cars are typically used. However, scrap cars are costly to acquire, and actual chemicals cannot be used for HazMat trainings because of safety and environmental concerns. With XR, trainings can be safely carried out with minimal setup. Training scenarios can also be customised to include different vehicles such as lorries and buses, which are not easily purchasable as scrap vehicles.

Other than NUS, HTX and SCDF also worked with RE-liON, a defence training software developer from the Netherlands, to develop a VR system that can now recreate and customise different building-fire scenarios.

Customisable realities with VR

In a computer-controlled environment, firefighters don a RedSuit that can emit heat like the MAV Suit. The gear comes with a mock-up of the firefighting nozzle which recreates the sensation of different water pressures, adding to the physical and virtual experience for trainees.

Components of a RedSuit. It has built-in sensors which track the location and movements of trainees in the virtual world. (Photo: HTX)

Training ground of SCDF officers in a VR simulation. (Photo: HTX)

Trainees can be exposed to multiple kinds of situations with varying degrees of unpredictability and severity. For instance, instructors can “plant” civilian characters in the building for rescue, or “reignite” fires that need to be decisively put out. Instructors can also observe the trainees from a floor-plan view or through their point of view (POV) to assess their situational awareness in real-time to give immediate feedback. The simulations are recorded so that they can be replayed to review the trainees’ performance again. 

Trainees are mapped as avatar characters on a computer screen where every action and movement can be detected. In this photo, an instructor watches the trainees navigate a room from a floor-plan view. (Photo: HTX)

A trainee’s POV as he goes through the VR training. (Photo: HTX)

A replay of a VR simulation set in a building on fire. (Photo: HTX)

A casualty spotted from the POV of a trainee. (Photo: HTX)

Efforts to make these training systems deployable have been underway since over a year ago. The XR system started at the end of 2021 while the VR firefighting system project started in March 2022. Both these systems are currently on trial and used for small group trainings. Once successful, SCDF is looking to scale them up for use in larger group training.

Can these technologies replace real-life training?

While VR and XR can provide the flexibility and ease of carrying out different firefighting training with enhanced realism, they are not here to completely replace hands-on training. Rather, these XR and VR systems will be used to supplement the usual theory and practical training of officers to enhance their tactical firefighting proficiency, situational response, and decision-making skills.

Dr Saravana Kumar Kumarasamy, the Deputy Director of Modelling & Simulation (M&S) at the Human Factors & Simulation (HFS) Centre of Expertise (CoE), heads these projects, and is realistic when it comes to using XR and VR for simulation training.

“We are not trying to replace conventional methods, but rather progressively raise the bar for simulation training,” he said. “It’s a step up from ‘classroom’ learning, but you cannot remove the field training for many years to come.”

“Still,” Dr Saravana continued, “there are obvious benefits to incorporating XR and VR into existing training. The systems allow officers to practise their skills in a controlled environment where they can experiment, learn, and adapt without being restricted by real-world limitations. We are continuously working with partners to enhance the situational realism in simulations, such as through the creation of novel senses and development of miniaturised devices that can be integrated into the headsets or haptic suits.”

Terence Teng, Lead Engineer at the HFS CoE, who is involved in these projects, added, “Trialling the different aspects of XR and haptic feedback helps us to find and gather the right technological pieces to build a complete XR training platform. This will not only revolutionise how we experience the virtual world through enhanced sensory fidelity, but also level up Home Team training.”

Lieutenant Colonel Foo Yiing Kai, Commander of Punggol Fire Station, says that the collaboration has been beneficial as it allowed each organisation to bring their unique skills, ideas, and perspectives together.

“It’s a system that will bring about greater training effectiveness and learning efficiency for my trainees,” he commented. “They are now better able to practice and refine their techniques and skills in a safe environment.”

We might still be quite a long way from XR games being available like VR or AR games, but by stepping up our Home Team trainings with these technologies, new and exciting possibilities are opening up for our officers so that they are ever-ready to safeguard our homeland.