Experiencing virtual worlds using immersive virtual reality (VR) technologies allows for a rich sensory experience. However, the hardware available for experiencing these virtual worlds is usually limited to addressing people in terms of their visual, auditory and haptic/tactile sensory modalities, via displays, headphones and vibrators in controllers. Addressing other sensory experiences (e.g. the sense of smell or pain) can usually only be achieved through elaborate, often prototypical, hardware designs, whose usability in private and scientific contexts with limited (spatial) resources is severely limited. Consequently, prototypical hardware constructions are not an adequate substitute for the majority of VR users.

A substitute for these elaborate hardware constructions can be the method of visual delegation. So-called visual delegates are visual effects and visual elements that replace sensations whose responsible sensory modalities are not addressed by the hardware usually used. Visual delegates delegate the generation of the sensations to be evoked to the visual sensory modality via the display to the user. Depending on the design of visual delegates, any number of complex and combined perceptions can be delegated.

Development of a framework

fig. 1: Visual Delegate Generalization Frame^[1]

Visual delegates have been used sporadically in prototypical VR applications and increasingly in video games. However, this use was usually less structured and based on common practices. Therefore, at the beginning of this research work, a structured and fundamental analysis of previously used visual delegates was conducted ^[2] in order to categorise them and to develop a catalogue of visual delegates with associated perceptual delegations. Furthermore, the term Visual Delegates was established with this work. On this basis, a framework was developed that allows Visual Delegates to be defined on the basis of visual properties in order to investigate Visual Delegates more specifically ^[3]. The effects of the individual properties of visual delegates on users were also analysed. Furthermore, concrete visual delegates were developed and their effectiveness was tested in concrete scenarios. Visual delegates for illness perception ^[4], age perception ^[5], distance perception ^[6] and stress perception ^[7] were examined with regard to their delegation ability. The results indicate that the perceptions investigated can be partially delegated by visual delegates, which suggests that the use of visual delegates in immersive and interactive virtual environments can promote a richer overall perceptual experience.

fig. 2: Experiencing Visual Migraine Impairments in Virtual Reality (IEEE SeGAH 2020) ^[8]

fig. 3: Intensifying Stress Perception Using Visual Effects in VR Games (ACM FDG 2020) ^[9]

References

1. Sebastian Misztal and Jonas Schild (2022): Visual Delegate Generalization Frame – Evaluating Impact of Visual Effects and Elements on Player and User Experiences in Video Games and Interactive Virtual Environments. In: CHI Conference on Human Factors in Computing Systems (CHI '22), April 29-May 5, 2022, New Orleans, LA, USA, pp. 20, ACM, New York, NY, 2022.
2. Sebastian Misztal and Guillermo Carbonell and Jonas Schild (2020): Visual Delegates - Enhancing Player Perception by Visually Delegating Player Character Sensation. In: Association for Computing Machinery, New York, NY, USA, 2020, ISBN: 9781450380744.
3. Sebastian Misztal and Jonas Schild (2022): Visual Delegate Generalization Frame – Evaluating Impact of Visual Effects and Elements on Player and User Experiences in Video Games and Interactive Virtual Environments. In: CHI Conference on Human Factors in Computing Systems (CHI '22), April 29-May 5, 2022, New Orleans, LA, USA, pp. 20, ACM, New York, NY, 2022.
4. Sebastian Misztal and Guillermo Carbonell and Lysann Zander and Jonas Schild (2020): Simulating Illness: Experiencing Visual Migraine Impairments in Virtual Reality. In: 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), IEEE, 2020.
5. Sebastian Misztal and Guillermo Carbonell and Jonas Schild (2022): Experiencing Age-Related Movement Impairment Through Visual Delegation in VR Can Substitute Haptic Impairments of an Age Simulation Suit. In: 2022 IEEE 10th International Conference on Serious Games and Applications for Health(SeGAH), 2022.
6. Sebastian Misztal and Guillermo Carbonell and Lysann Zander and Jonas Schild (2020): Jumphair: Improving Jumping Performance in First-Person Video Games Through Visual Assistance. In: International Conference on the Foundations of Digital Games, ACM, 2020.
7. Sebastian Misztal and Guillermo Carbonell and Lysann Zander and Jonas Schild (2020): Intensifying Stress Perception Using Visual Effects in VR Games. In: ACM International Conference Proceeding Series, 2020, ISBN: 9781450388078.
8. Sebastian Misztal and Guillermo Carbonell and Lysann Zander and Jonas Schild (2020): Simulating Illness: Experiencing Visual Migraine Impairments in Virtual Reality. In: 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), IEEE, 2020.
9. Sebastian Misztal and Guillermo Carbonell and Lysann Zander and Jonas Schild (2020): Intensifying Stress Perception Using Visual Effects in VR Games. In: ACM International Conference Proceeding Series, 2020, ISBN: 9781450388078.