Campers at the “Big Break: Singing and Dancing” camp play Big Break: The Acting Game
TELUS is always looking for innovative ways to enhance communication.
That’s why when Ryerson University applied for a TELUS Greater Toronto Area Community Board grant for the development of an app that could improve the social skills of youth living with autism, the decision was overwhelmingly in the affirmative.
Big Break: The Acting Game is an educational app that supports social communication skills in children and teens. Designed in the Ryerson SMART Lab, and currently available for iPad platforms, gameplay involves first viewing and then imitating emotional songs performed by actors. Using facial and vocal tracking technology, the app records the player’s intonation and facial expressions during the song. Big Break then provides a score and feedback based on how closely the user matched the actor’s performance. Other features include modules requiring classification of emotion, and practice with scaling the intensity of emotion.
TELUS’ contribution supported program design and delivery of the app, its development, and the creation of a program guide. Additional research and development support was provided by the federal and provincial government.
”Our philosophy is ‘we give where we live,’” said Jennifer Kirner, senior manager, community affairs at TELUS. “This project touched on health, education, and the innovative use of technology – all key areas we focus on for a community investment. We’re really looking forward to learning about how the cohort benefitted from the app, and the social impact it will have on the community.”
The game is the brainchild of Lucy McGarry, a 2014 Ryerson PhD graduate in psychology research. Her doctoral thesis looked at how mimicking voice and facial expressions could improve communication skills in people with autism. She designed Big Break as part of her dissertation, collaborating with neuroscientist Frank Russo, a psychology professor and director of the SMART Lab at Ryerson. To develop the game as an app, the team co-founded the social enterprise, Emote Play Apps.
“Neuroplasticity research suggests that regular practice and personalized feedback are important determinants of successful learning,” said McGarry. “In Big Break, we have incorporated carefully designed music, sound effects, graphics, and interactive elements to enhance game play. Sophisticated algorithms analyze expressions and provide immediate feedback. Players also get to see themselves in short movies, providing the player with unique insights.”
Although the game can be enjoyed by anyone, it is ideal for children with communication difficulties, including children with autism spectrum disorder (ASD). Most individuals with ASD face challenges in typical social interactions. Part of this is due to an impediment in some children with autism to simulate the body language and vocal cues typical in conversations.
“You can have insight into a person’s intention by simulating – not necessarily overtly - what they’re doing,” said Russo. “It’s not the only way we perceive emotion, but it’s an important way. We know that this simulation is not functioning typically in autistic children. We believe that practice with this system is going to help rehabilitate the network that’s involved with identifying emotions. And we think that incorporating music might be helpful as it creates a temporal framework that may facilitate simulation,” he added.
A previous study using the app revealed that 13-weeks of game play involving simulation led to improvements in social-emotional relations in comparison to a control group that used the game without simulation. Given the data, Russo and McGarry successfully pitched an idea to the Geneva Centre for Autism for a music and drama camp using Big Break, leading to two week-long sessions in July and August. Sixteen campers were hosted on the Ryerson campus. The camp involved choir singing, rhythmic games, acting, and exercises designed to provide practice with imitation of vocal, facial, and gestural expression. The activities were further augmented by the technology assisted feedback provided through the app. The results again returned positive, and the Geneva Centre is now standardizing the delivery of the camp using Big Break, creating a package that other centres can use to achieve comparable outcomes.
For more information about how to support Ryerson, please visit the Ryerson Supporting website.