Adapting a QA Lab for Virtual Reality

 

VR is changing the QA process with its unique requirements. We asked David Kilgour, VMC’s Global Platforms Test Manager, to tell us some of the factors VMC took into account when expanding its VR test facility.

Since VR and AR began showing commercial viability, there have been parallel discussions about both the potential and challenges of the technology. Developers have steadily improved the virtual experience, yet factors such as headset prices, motion sickness, and limited application outside of gaming have been obstacles to widespread adoption. Now, as the industry focuses on perfecting the visual and auditory experience, the user base has grown exponentially, and this is driving more industries to explore VR’s potential, including entertainment, education, online shopping, fitness, and eager marketers in every industry.

QA will be a critical factor in the growth of both VR and AR, and while the goals are the same – get the highest-quality product to market as quickly as possible – there are several key differences between VR testing and traditional console, PC, and mobile-based QA. Some are obvious, others are subtle, but they were all important considerations as we expanded our VR test environment.

Space

While traditional QA can be squeezed into nearly any available space, VR testing needs a lot of room. Testers will be moving around with no sense of the physical space around them. What might be simple for a regular tester to notice, a VR tester may not. This extends beyond the configuration of the furniture and into more daily tasks: if IT is working on an issue for the adjacent tester, the extra people and items are easy for traditional testers to notice but become a hazard for VR testers who, even if they know the extra items are there, may lose track when they’re deep in testing. To maintain a safe working environment, everyone has a responsibility to ensure that the environment remains clear during testing.

Sunlight

I’m not saying that all VR testers are vampires – but imagine for a moment they are. In a more traditional environment, window blinds and lighting would be adjusted throughout the day so that testers have sufficient light (artificial and natural) without direct sunlight on their monitors. VR testers can be immersed in their testing environment for hours without any sense of changes to their real environment, so they could remove their headset to face harsh, direct sunlight. This wouldn’t be good for a vampire or for the VR tester. With this in mind, we plan the space so that the team gets the right amount of light without fear of direct sunlight.

Motion sickness

VR motion sickness is a real thing, and because some people experience it more than others, screening testers is important. It doesn’t matter how effective the tester may be, if they get sick after wearing the headset for 30 seconds (our record is three seconds) then they’re not suitable for VR. Even for those who are suitable, most people get a little motion sick after testing for an extended time, so we provide ginger-based items (ginger candy, ginger ale, ginger tea, etc.) because it helps combat VR-induced motion sickness. Testers can also take breaks as they feel they need instead of being tied to a set break pattern, with some testers taking a few minutes every hour while others preferring a longer break after multiple hours of testing.

Documentation (Test Plans, Walkthroughs, Text Files, etc.)

VR testers are immersed within their environment, so any documentation is difficult to reference within the game, especially a complex and time-critical walkthrough document. Steps are easily missed if a test plan has very specific and cumbersome testing path, and scrolling text is difficult to check against a text file. The easiest way to adjust for many of these issues is having someone available for the tester to talk to. It could be as simple as having a second tester read the test plan as the VR tester moves through the virtual word, and having the VR tester read aloud the text they see and having a second tester check it against the text file.

Genre

With traditional QA, the genre of a title is less of a priority than a tester’s experience on a particular console, but in the VR environment, genre needs to be considered when selecting the right team. Consider the scariest horror movie you’ve seen – now turn that into a video game and place yourself in the middle of it with your VR headset. While it might be amusing to have someone on the team scream the first time a ghost or monster appears in front of them, it can become quite distracting if it was to happen all the time. Before we assign a test team, we speak with each tester individually and give them an overview of the game. If the genre isn’t suitable, we find testers who are more interested.

Potential downside

With every new technology, there is always a downside. We joke that with VR, it is difficult to tell the difference between a tester who is concentrating on one particular aspect and one who has simply fallen asleep.

We Grow as VR Grows

VMC’s VR testing facility is very different from the area we started with a year ago, and as VR technology evolves and player expectations change with it, we’ll continue to make improvements to both our VR testing methods and our space.

VMC’s expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com

Adapting a QA Lab for Virtual Reality

The Power of Play – April 26th, 2017

One Day. One Industry. Infinite Fun.
The Power of Play Countdown Begins!

 

Join VMC at THE game industry event in the Seattle area on April 26th! Hear from Bungie, Blizzard, Xbox and more! As a sponsor of this fabulous event, we are providing a discount link for 30% off so you can join us: https://www.eventbrite.com/e/power-of-play-2017-registration-29453983658?discount=ARFGames

From the biggest companies like Xbox, Bungie, Blizzard, and Oculus to the best of the indies like Ark Survival, Oxenfree, and Darkest Dungeon, all will be at Power of Play do some sharing, learning and celebrating.

On Wednesday at the Meydenbauer Center in Bellevue you’ll hear top industry speakers, mingle at the award reception, and see the newest indie and VR games around.

For more information go to: www.powerofplay.us. For more information on VMC or to set up a meeting with us, please email info@vmc.com.

At Power of Play You’ll:

* Learn from some of the biggest names in the business like Xbox, Blizzard, Wizards of the Coast, Oculus just to name a few.

* Hear VR experts discuss the near-term state of the market. What will be the single most under-tapped utility in VR? What is being done to address this? Where has VR delivered above or below expectations?

* Watch the top eight finalists present in front of our industry judges as they compete for the Seattle Indie Game Competition crown and $2,500!

The Power of Play – April 26th, 2017

VR is a reality at VMC


VR has arrived. Amy Nanto (Sr Manager, Business Development, Games) highlights a few things to think about when creating your immersive experience.

In the 90’s, the companies creating virtual reality content had lofty sci-fi aspirations but were limited by technology. We now live in the futuristic world where you can strap on a VR headset and immerse yourself in a truly convincing environment. The creative goal for VR focuses on exceptional experiences – whether it’s games, entertainment, sports, or journalism – but there are some important considerations to ensure the user experience is safe, believable, and truly immersive.

Platforms

People say content is king, but the platform also rules. Content creators need to consider how their experiences will work on the growing number of VR platforms, headsets, and peripherals vying to establish themselves in the VR landscape. A new era of VR has begun, and with the continued introduction of new headsets and software, VR devs need to ensure their content works smoothly on the latest SDKs.

Standards

Even the most talented games professionals can face development challenges getting their VR content to comply with all standards and industry best practices for safety and comfort. Some of these challenges include minimal experience performing thorough QA tests on VR experiences, limited access to hardware for QA and development, and insufficient information on best practices for start-to-finish VR development.

Quality Assurance

Proper testing of your content is absolutely crucial. Testing VR content presents unique challenges when compared to traditional content because the user is completely immersed within the VR world. Low latency, high frame rate, high resolution, directional audio, and good calibration are essential to creating a plausible reality, and a lapse in any factor can take the player out of that virtual world. Knowing how to QA immersive VR environments is vital to ensuring optimal player experience.

Experience

After working closely with various VR teams for almost a year before VR devices became commercially available, VMC has developed an in-depth understanding of VR platforms on mobile and PC. Our test capabilities include functionality, usability, compatibility, and end-to-end localization on both VR content and hardware in our dedicated VR testing lab.

Make Something Awesome

Whether you’re creating VR for mobile, PC, console, or Cardboard, now is an exciting time to be a pioneer in VR. Go make something awesome!

VR is a reality at VMC