Play to improve: Gamifying usability evaluations in virtual reality
Design research, space, interaction and service design - Enhancing the productivity and quality of the design operations process.
​
Miglani A., Kidambi S., Mareguddi P (2020). Play to improve: Gamifying usability evaluations in virtual reality. Proceedings of HCI International, Copenhagen, 2020 [Link]
​
Tools used: Unity, C#, Oculus rift, Blender, SPSS
Group size: 3
TASK
-
Study the 'as is' business process and generate a service blueprint
-
Identify potential areas of improvement and generate 'to be' service blueprint
-
Frame research questions and hypothesis
-
Design and develop proof of concept
-
Designing the experiment and user testing for checking the hypothesis framed
BUSINESS VALUE PROPOSITION
When performing usability design ​
activities, it was observed that the entire way of working for design operations could be improved.
​
All usability designers in the Usability Design Centre of Excellence community were consulted to know their challenges and motivations.
​
A business value proposition canvas was framed to come up with different solutions in which the design delivery and operations process could be improved.
CONTEXTUAL ENQUIRY
One of the pains pointed by usability designers was the difficulty and the need of simulating the actual field environment while conducting usability design activities in lab setting.
Contextual enquiry studies were conducted at a local diagnostic imaging clinic in which workflow centring radiologists, technologist and administrators was observed. This study also included studying the environment and comparing it with the lab environment.
STORYPLY IDEATION
Post field visits, a 'storyply' workshop was conducted wherein designers were asked how can we utilize virtual/augmented reality technology further for the heathcare and usability design domain.
​
Going by the purpose for which the storyply innovation tool is designed, 3-4 ideas emerged which were incorporated in the prototype. Like for instance, one idea suggested to include the demonstration of the latest product release changes. This idea when clubbed with the original idea of simulating the field environment in virtual reality, gave rise to altogether a new idea of gamifying the entire usability design activity.
​
The usability designing simulation tool/ game can only be incorporated in the regular agile release train and put to use in the regular product development cycle (or for mass acceptance) after careful transition from physical or lab environment. For this environment transition, design research activities were performed for which different research questions and hypothesis were tested.
RESEARCH QUESTIONS
-
How influencing a physical environment changes usability results?
-
How are physical environments different from virtual environments to check usability of user interfaces?
-
What interaction modality in virtual environment would give similar experiences as in physical environment?
-
What combinations of interactions would have the best usability (and could replace the conventional interaction modalities)?
-
How are parameters – Usage environment, task performance, usage behavior, design intuitiveness – Environment and UI related to each other?
-
Can Usability Engineering for SAMD be done using virtual reality/augmented reality technology? [FDA]
-
What all factors influence the environment and ultimately immersion in virtual reality?
PROTOTYPE DESIGN & TEST
A prototype/ proof of concept was designed keeping product design and interaction design principles into consideration. ​
​
This prototype aimed at answering the research questions and testing the research hypothesis.
​
For designing and developing the prototype, a design strategy was framed which listed out the steps which should be followed to result in 3 interaction concepts as pointed out in the research hypothesis.
​
A game containing 3 sets with 3 stages each was designed. These 3 sets are designed for each interaction concept to be evaluated. Whereas each set having 3 stages - introduction, performance and exit is the way the user (here acting like a technologist) would be participating in the usability design experiment.
​
Furthermore, an experiment design framework was designed to test the research hypothesis.
​
RESULTS
PRESENCE
(IPQ)
USABILITY
(SUS)
WORKLOAD/ EFFORT
(NASA TLX)
PREFERENCE RANK
QUALITATIVE DATA
(WORD CLOUD)
DISCUSSION
For virtual environments, system usability scores (SUS) for the three interaction concepts designed namely, gaze timer, gaze click and gaze gesture had mean scores falling below the threshold value of 68. This shows that virtual environment does effect the usability when virtual environment interaction concepts are used (as compared to mouse or keyboard used in physical environment).
​
Moreover, positive correlations were found between gaze gesture concept and involvement construct (spearmann rho – 0.723, p-0.043) and experienced realism construct (spearmann rho – 0.704, p -0.051).
Gaze gesture concept having lowest usability, had highest involvement and lowest experienced realism as compared to the other two interaction concepts.
For preference, participants preferred concept B: gaze click the most.
Evidently, participants prefer to have control on the actions they are performing – similar to mouse and keyboard.
Moreover, RULA scores also showed an acceptable posture for the concept. This is also in line with the usability scores (highest), mental demand scores (lowest), physical demand scores (lowest), temporal demand scores (lowest) and effort scores (lowest).
​
When making a transition from a physical environment or a conventional lab environment to having a virtual environment, gaze click concept can be used to control the workflow of pages. This guideline can be used further by other researchers working on usability engineering in virtual reality theme.
SERVICE BLUEPRINT
The service blueprint for both 'as is' and 'to be' situations is shown above. 'To be' situation is with introduction of gaze click gesture in virtual reality system to evaluate usability of software designs.