VR is changing the QA process with its unique requirements. We asked David Kilgour (Global Platforms Test Manager) to tell us some of the factors VMC took into account when expanding its VR test facility.
One of the keys to testing virtual reality is the need to pay close attention to the physical environment the tester blocks out when they wear a headset. While the QA goals are the same – get the highest-quality product to market as quickly as possible – there are several key differences between VR testing and traditional console, PC and mobile based QA. Some are obvious, others are subtle, but they were all important considerations as we expanded our VR test environment.
While traditional QA can be squeezed into nearly any available space, VR testing needs a lot of room. Testers will be moving around with no sense of the physical space around them. What might be simple for a regular tester to notice, a VR tester may not. This extends beyond the configuration of the furniture and into more daily tasks: if IT is working on an issue for the adjacent tester, the extra people and items are easy for traditional testers to notice but become a hazard for VR testers who, even if they know the extra items are there, may lose track of those items when they’re deep in testing.
I’m not saying that all VR testers are vampires – but imagine for a moment they are. In a more traditional environment, window blinds and lighting would be adjusted throughout the day so that testers have sufficient light (artificial and natural) without direct sunlight on their monitors. VR testers can be immersed in their testing environment for hours without any sense of changes to their real environment, so they could remove their headset to face harsh, direct sunlight. Both the reorientation to the real world and sudden bright light can be very unpleasant, so we’ve planned our space to minimize this experience.
VR motion sickness is a real thing, and because some people experience it more than others, screening testers is important. It doesn’t matter how effective the tester may be, if they get sick after wearing the headset for 30 seconds (our record is three seconds) then they’re not suitable for VR. Even for those who are suitable, most people get a little motion sick after testing for eight hours, so we provide ginger-based items (ginger candy, ginger ale, ginger tea, etc.) because it helps combat VR-induced motion sickness. Testers can also take breaks as they feel they need instead of being tied to a set break pattern, with some testers taking a few minutes every hour while others preferring an extended break after multiple hours of testing.
Documentation (Test Plans, Walkthroughs, Text Files, etc.)
VR testers are immersed within their environment, so any documentation is difficult to reference within the game, especially a complex and time-critical walkthrough document. Steps are easily missed if a test plan has very specific and cumbersome testing path, and scrolling text is difficult to check against a text file. The easiest way to adjust for many of these issues is having someone available for the tester to talk to. It could be as simple as having a second tester read the test plan as the VR tester moves through the virtual word, and having the VR tester read aloud the text they see and having a second tester check it against the text file.
With traditional QA, the genre of a title is less of a priority than a tester’s experience on a particular console, but in the VR environment, genre needs to be considered when selecting the right team. Consider the scariest horror movie you’ve seen – now turn that into a video game and place yourself in the middle of it with your VR headset. Before we assign a test team, we speak with each tester individually and give them an overview of the game. If the genre isn’t suitable, we find testers who are more interested in that title.
With every new technology, there is always a downside. We joke that with VR, it is difficult to tell the difference between a tester who is concentrating on one particular aspect and one who has simply fallen asleep.
We Grow as VR Grows
VMC’s VR testing facility is very different from the area we started with a year ago, and as VR technology evolves and player expectations change with it, we’ll continue to make improvements to both our VR testing methods and our space.