Supporting Supporting Bodily Communication in Video Consultations of Physiotherapy


Located within the Human Computer Interaction (HCI) tradition, I am investigating the role of bodily communication in video based clinical consultation that connects remote patients and clinicians over video. Bodily communication is defined as non-verbal signals such as eye gaze and face expressions that are communicated through our body during social encounters. Prior works define bodily communication as a crucial parameter of face-to-face clinical consultations where clinicians and patients use nonverbal cues to establish rapport and to communicate details of the health issue. However, there is a lack of understanding on how bodily information is communicated during video consultations, particularly, in domains like physiotherapy where the diagnosis and treatment are centred on body movements and functioning. This work endeavours to help identify and address these gaps.

The main research question that I am exploring in my thesis is: how can interactive technologies support physiotherapists in assessing bodily information during video consultations? I have answered this question through three studies. The first study investigates the challenges that physiotherapists face in interpreting the bodily information in current practises of video consultations. This study involved observations of 10 naturally occurring face-to-face and video consultations of physiotherapy at Royal Children’s Hospital, Melbourne. The study highlighted that video technology does not communicate a variety of essential bodily cues throughout the video consultation. Consequently, the treatment offered by physiotherapists in video consultations was less specific than face-to-face consultations. The study also opened up design opportunities that I harnessed to design a research prototype in the second phase.

To support the lower limb assessment in video consultations, I developed a wearable technology, SoPhy. SoPhy is a wearable technology consisting of a pair of socks with embedded sensors for patients to wear; and a web interface that displays information about range of weight distribution, foot movement, and foot orientation for physiotherapists in real-time. The design phase was followed by a laboratory study where I evaluated the utility of SoPhy in enhancing the diagnostic confidence of physiotherapists during video consultations. I simulated the setting of video consultations across two rooms where postgraduate physiotherapy students acted as physiotherapists and patients. I conducted 40 video consultations, half each with and without SoPhy, for patients with lower limb issues. Findings showed that SoPhy increased the confidence in assessing squats exercise and fewer repetitions were required to assess patients when using SoPhy.

After examining the potential of SoPhy in supporting physiotherapists, in the final study, I am investigating the use of SoPhy in naturally occurring video consultations of physiotherapy. The study involves field deployments of SoPhy with (real) patients and physiotherapists at Royal Children’s Hospital. I will simulate the setting of video consultations in two rooms at the hospital where physiotherapists and patients will communicate with each other via a video conferencing tool.

Here is a brief video on the working of SoPhy during video consultations

Go through the slides for a quick overview: