Away from the glowing rectangles

From news stories to novels, from HR reports to history books, from data sets to dating sites, human beings create and consume almost every kind of communication the same way: seated in front of a glowing rectangle. Screens seem inexorably to have subsumed all previous media, ushering in a new age.

Alexandra Mazalek has long been dissatisfied with the dominance of screen-mediated experience. Screens limit human capacity to understand and interpret information, she says, and also constrain the kinds of stories we tell. “I started thinking about ways I could experience stories that didn’t involve a box sitting on a table in front me,” says Mazalek, an RTA School of Media professor. “That took me toward what’s now known as tangible and embodied interaction.”

A Canada Research Chair in Digital Media and Innovation, Mazalek runs Ryerson’s Synaesthetic Media Lab (Synlab), where she develops and experiments with tactile, physical tools that can interact with digital environments and information.

On the science side, that can mean using so-called “active tangibles” – handheld devices that can sense and be sensed by one another. These tangibles can also store data and display it graphically using a table-sized touch-sensitive screen.

In a prototype, one tangible was loaded up with genetic data, and a second with related clinical data collected from cancer patients. By touching, stacking, sliding, bumping and rotating these tangibles, scientists could interact physically and digitally with multiple datasets, revealing unexpected correlations between genetic mutations and physical conditions.

Mazalek says physical interaction can make a huge difference on the kinds of information people draw from data.

 “When Francis and Crick built a physical model of DNA, it opened up whole new insights and streams of research,” she says. “We want to find ways to give abstract data sets some sort of physical form.”

Working with geneticists, biochemists and other scientists, her team develops interfaces that draw on Synlab’s fundamental research into spatial cognition – the study of how people form mental models of 3D objects and space.

That spatial cognition research also feeds into other projects, including museum partnerships aimed at telling cultural stories in new ways.

In 2014, the Robert C. Williams Paper Museum in Atlanta staged an exhibit called “Mapping Place: Africa Beyond Paper” that included a section on a type of African map known as a Lukasa board. Instead of compass points and scale-drawn coastlines, Lukasa boards feature beads and carvings that help the user recall stories and songs that describe local history and geography.

It takes specialized knowledge to use a traditional Lukasa board. Using a variation on their tangibles devices, Mazalek’s team created a digital and physical interface that brought the experience alive for museum visitors in a way that would have been impossible with standard touch screens.

 Mazalek pursues many research directions simultaneously. She says projects like the Lukasa have been deeply satisfying, and that she is still striving to produce similar breakthroughs on the science side.

“Whatever dataset we’re working with in whichever scientific discipline, my ultimate goal is for our tools to help someone create a new insight or discovery,” she says. “Then I feel that we’ve succeeded.”