VR is changing the QA process with its unique requirements. We asked David Kilgour, VMC’s Global Platforms Test Manager, to tell us some of the factors VMC took into account when expanding its VR test facility.
Since VR and AR began showing commercial viability, there have been parallel discussions about both the potential and challenges of the technology. Developers have steadily improved the virtual experience, yet factors such as headset prices, motion sickness, and limited application outside of gaming have been obstacles to widespread adoption. Now, as the industry focuses on perfecting the visual and auditory experience, the user base has grown exponentially, and this is driving more industries to explore VR’s potential, including entertainment, education, online shopping, fitness, and eager marketers in every industry.
QA will be a critical factor in the growth of both VR and AR, and while the goals are the same – get the highest-quality product to market as quickly as possible – there are several key differences between VR testing and traditional console, PC, and mobile-based QA. Some are obvious, others are subtle, but they were all important considerations as we expanded our VR test environment.
While traditional QA can be squeezed into nearly any available space, VR testing needs a lot of room. Testers will be moving around with no sense of the physical space around them. What might be simple for a regular tester to notice, a VR tester may not. This extends beyond the configuration of the furniture and into more daily tasks: if IT is working on an issue for the adjacent tester, the extra people and items are easy for traditional testers to notice but become a hazard for VR testers who, even if they know the extra items are there, may lose track when they’re deep in testing. To maintain a safe working environment, everyone has a responsibility to ensure that the environment remains clear during testing.
I’m not saying that all VR testers are vampires – but imagine for a moment they are. In a more traditional environment, window blinds and lighting would be adjusted throughout the day so that testers have sufficient light (artificial and natural) without direct sunlight on their monitors. VR testers can be immersed in their testing environment for hours without any sense of changes to their real environment, so they could remove their headset to face harsh, direct sunlight. This wouldn’t be good for a vampire or for the VR tester. With this in mind, we plan the space so that the team gets the right amount of light without fear of direct sunlight.
VR motion sickness is a real thing, and because some people experience it more than others, screening testers is important. It doesn’t matter how effective the tester may be, if they get sick after wearing the headset for 30 seconds (our record is three seconds) then they’re not suitable for VR. Even for those who are suitable, most people get a little motion sick after testing for an extended time, so we provide ginger-based items (ginger candy, ginger ale, ginger tea, etc.) because it helps combat VR-induced motion sickness. Testers can also take breaks as they feel they need instead of being tied to a set break pattern, with some testers taking a few minutes every hour while others preferring a longer break after multiple hours of testing.
Documentation (Test Plans, Walkthroughs, Text Files, etc.)
VR testers are immersed within their environment, so any documentation is difficult to reference within the game, especially a complex and time-critical walkthrough document. Steps are easily missed if a test plan has very specific and cumbersome testing path, and scrolling text is difficult to check against a text file. The easiest way to adjust for many of these issues is having someone available for the tester to talk to. It could be as simple as having a second tester read the test plan as the VR tester moves through the virtual word, and having the VR tester read aloud the text they see and having a second tester check it against the text file.
With traditional QA, the genre of a title is less of a priority than a tester’s experience on a particular console, but in the VR environment, genre needs to be considered when selecting the right team. Consider the scariest horror movie you’ve seen – now turn that into a video game and place yourself in the middle of it with your VR headset. While it might be amusing to have someone on the team scream the first time a ghost or monster appears in front of them, it can become quite distracting if it was to happen all the time. Before we assign a test team, we speak with each tester individually and give them an overview of the game. If the genre isn’t suitable, we find testers who are more interested.
With every new technology, there is always a downside. We joke that with VR, it is difficult to tell the difference between a tester who is concentrating on one particular aspect and one who has simply fallen asleep.
We Grow as VR Grows
VMC’s VR testing facility is very different from the area we started with a year ago, and as VR technology evolves and player expectations change with it, we’ll continue to make improvements to both our VR testing methods and our space.
VMC’s expertise ensures the most innovative companies in the world deliver an excellent product experience to every customer, everywhere. Learn more at vmc.com.