By Scott Smith
Sharing info across devices with the wave of a hand may be new to Microsoft, but we were applying it to telehealth years ago.
Is Microsoft’s product research following in our footsteps? The tech giant introduced at UIST2021 a product called AirConstellations that performs “spatially-aware cross device interactions,” which includes gestures to perform certain functions and enhanced sharing between screens. Back in the late `90s, Shaun Illingworth and I were developing this type of gesture-based user interface and multi-device integration for early telemedicine projects.
Our design was patented in 1998, as was a concept for collaborative shared space in 1999.
We’re not calling foul here, just pointing out that as behavioural researchers, we long ago recognized the value of using multiple devices to provide a more natural interaction. In our case, it was for healthcare professionals working remotely with patients. Giving a practitioner easy access to files and charts during video consultations (via touchscreens and gestures) supports the development of a rapport between the patient and the service provider.
Here are Shaun (operating the video conference) and I (the patient) in Pembroke General Hospital in the late `90s. This was the current technology and implementation at the time. Substantial cost savings was achieved by treating patients on site, while cardiologists were able to log on remotely. Remote consultation saved an $8,000 hospital visit.
Using multiple devices improves remote healthcare
Shaun and I were part of the team that developed several patents related to the use of multiple devices to create a “shared space” and gesture-based controls for video conferencing with patients. The concept of shared space as we envisioned it in 1999 is shown in the diagram.
Telehealth has become the norm during COVID-19. But when we first embraced the concept, it was essential to investigate how telecommunications technology could enhance healthcare, considering the human factors and behaviours.
We talked to hundreds of healthcare professionals
In one project, I managed qualitative research, workshops, and patient needs analysis for homecare solutions for hospitals in New Brunswick. The outcome of this research was a homecare device and care-delivery method for cardiac patients to stay in touch after their recovery from surgery.
I also led teams conducting design research throughout the North American medical community into telehealth and homecare needs. Over the years, the design teams under my direction interviewed more than 200 physicians, 180 healthcare administrators and 500 patients.
Now, in 2021, even kindergarten kids can use video-conferencing software, but in the late `90s that was not the case. Shaun and I researched and defined behavioural principles for remote consultation. We produced leading-edge concepts in telehealth, based on iterative design interviews with medical professionals.
How do you build trust in virtual consultations?
One concern, then and now, involves trust. How does the patient establish trust in the doctor? How does the doctor feel assured that the patient is giving an accurate, complete story? Furthermore, how do we support that need for trust in design?
We found natural interaction is one key to building trust.
Another thing that’s important for trust is competency. Good telehealth design will hide the technology and complexity, and let the doctor simply demonstrate his or her conclusions. It will amplify the doctor-patient relationship.
An early device that we called the Shared Space Monitor allowed the physician to interact naturally with relevant data and artifacts. Physicians want to interact with patients, not technology. So, the design of remote healthcare devices and services should minimize interference with the doctor-patient relationship.
Through our research, we have learned a lot about service design for healthcare. Enough that even Microsoft is following our lead.
