AI-enhanced therapeutic virtual reality programs

The immersive nature of virtual reality (VR) programs, enhanced with artificial intelligence (AI), is being leveraged to develop digital therapy tools that respond and adapt to users’ needs. 

Toronto Metropolitan University (TMU) professor Naimul Khan, cross-appointed to the Faculty of Engineering and Architectural Science and The Creative School, is the director of the TMU Multimedia Research Laboratory. He creates AI-enhanced VR tools that support mental health and are adaptive based on physiological signals, such as heart rate detection. 

“Digital therapy can help create pathways to treatment approaches that are both scalable and accessible,” said professor Khan. “Patients can utilize digital therapy tools in their homes, while clinicians can monitor patients remotely through AI-backed analytics.” VR offers users a sense of immersion and convenience, which could help address delays in accessing health-care services caused by issues such as pandemic-related backlogs.

Professor Khan said creating an AI model that can assess a person’s stress levels or emotional state can be “tricky.” His research team has been collecting data since 2018, including using electrocardiogram (ECG) readings from sensors worn, like a smart watch, to build increasingly accurate models that classify stress levels. One of the team’s recent models achieved 72.7 per cent accuracy and outperformed every related published study. The team is adding eye-tracking through a VR headset to their collected physiological data to further improve performance. 

Professor Khan has a number of projects relating to digital therapeutic tools.

Therapeutic games

The VR game “Subway Sensations,” offers children with autism the opportunity to experience riding the subway virtually and assess what about the experience creates stress. By adapting the game content to the user through AI, clinicians can view and compare data with previous game sessions through the dashboard that professor Khan and his team developed. This project will undergo a usability study in early 2024 and continues professor Khan’s work on therapeutic VR games that began with his collaboration with Holland Bloorview Kids Rehabilitation Hospital and entertainment company Shaftesbury’s previous technology division, which is now owned by UNLTD Inc. 

As part of this collaboration, professor Khan and his lab also tested the game “Bubble Bloom,” which was developed by Shaftesbury’s former technology division as a positive distraction game to help children cope with stress. His team’s research focused on adult users and found that the game had a positive impact on stress reduction. The lab also used this game to collect data as part of their experiments in classifying stress levels using sensor data that gave feedback on the player’s heart rate, skin response and respiration.

Digital therapy can help create pathways to treatment approaches that are both scalable and accessible.

Watch professor Khan's recent plenary talk (external link)  about his virtual reality and AI research at the University of Windsor. He begins speaking at about the three minute mark. 

Read “Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion” (external link)  in IEEE Sensors Journal. 

Read “Physiological Signal Analysis and Stress Classification from VR Simulations Using Decision Tree Methods” (external link)  in Bioengineering. 

Professor Khan’s research is supported in part by the New Frontiers Research Fund, the Natural Sciences and Engineering Research Council of Canada, National Research Council Canada, and UNLTD VR.