One of the challenges in the ViTAWiN research project was the integration of emotions in medical VR training. In previous studies, participants complained about the lack of realistic patient reactions. The burn patient did not behave like someone who was in terrible pain. Auditory cues were not available and the patient's pain could only be inferred from the symptoms.

Emotions were integrated with a processed patient whose facerig allows emotions to be reliably and realistically displayed. These emotions are supplemented with auditory feedback so that the virtual patient has a higher degree of realism and credibility.

Focus group with the education partners

In order to create the foundation for the integration of emotional expressions, a focus group was conducted. In it, an interdisciplinary exchange took place with trainers of the vocational study for paramedics, advanced training for emergency nursing and a medical education expert. They were conducted from one human-computer interaction researcher (Guillermo Carbonell) and one interaction designer (Madeline Ebeling).
The participants rated the relevance of emotional expressions and reactions as very important. They described them as a source of information for the medical staff who are supposed to treat the virtual patient.

fig. 1: Emotions at treatment

They named pain as the most important expression. They have the patients rate their pain on a scale of one to ten. One is a mild pain and ten is the strongest pain they can imagine. This would be permanently present during the treatment. It could increase, for example, when the wound is touched, or decrease due to the injection of painkillers. The emotions fear, shame or relief could be present as reactions to the treatment by the medical staff.

Online study with face expressions

In an online study, we examined our designs of emotional expressions. Several representations of pain were created with a pain score of one to ten, a representation of fear, shame and relief. Among other things, these were evaluated and ranked by the participants in various tasks according to their credibility and realism.

fig. 2: Pain expressions in the online study

In media design, the portrayal of emotions is considered a particular challenge, as the aim is to replicate the human face. The effect of the Uncanny Valley can occur, which describes a degree of realism in the representation that is classified as unrealistic by recipients and is therefore not accepted.
Paul Ekman's FACS offered a concept according to which expressions could be recreated as naturally as possible. Therefore, we chose a 3D character for the online study whose facerig was created according to the FACS and were thus able to avoid the Uncanny Valley.

fig. 3: Expressions of relief, shame and fear in the online study

Emotional interaction concept

Based on the Focus Group and the online study, we developed an interaction concept to define when and which emotion should be triggered for the virtual burn patient. Initially, pain, fear and relief were integrated, although other emotions are conceivable.
Injecting the patient with a syringe causes fear. If the medical staff removing the patient’s clothes, shame as emotion will be triggered. Touching a burnt area accidentally activates a brief severe pain. Relief will be triggered by touching an area that was not compromised causes for a brief moment, while similarly administration of morphin and a relatively long eye contact with patient during communication causes a stronger relief.

fig. 4: Interaction causes pain