Mobile QA with Accessibility in Mind

Jonathan Fournier, Mobile Test Lead at VMC, discusses the value and scope of Accessibility Testing for mobile games and apps. 

The World Health Organization estimates that 15% of the world population lives with some form of disability. This is a significant demographic of mobile device users, making accessibility an important factor in mobile game and app design. Mobile tech giants like Apple and Samsung already see the value in making sure their devices are inclusive, adding an impressive and growing array of accessibility features to their devices over the years. It would benefit mobile game and app developers to bolster this trend by considering accessibility during the development of their products.AccessPic

At VMC, we’ve put great effort into identifying how games and apps can be optimized in terms of accessibility. As part of our full range of QA services, we offer developers focused accessibility testing as a service, examining games and apps with attention to the following factors:

Limited Vision (including Color Perception)
Are text and images as clear as they can be? Is the color contrast suitable? Are there audio cues to help users along? Does the app interact well with text-to-speech apps or hardware?

Limited Hearing
Are the audio elements of your app mixed well? Is the volume acceptable? Is there anything important missed if the user opts to play with no sound? Does the app connect to external speakers, hearing aid, etc.?

Limited Vocal Capacity
If you allow voice prompts from users, are there alternative ways to give input? Does the app work with external mics or text-to-speech apps/hardware?

Limited Dexterity
How dependent is your app on actions like repeated tapping, holding a finger to the screen for long periods or overly precise swiping? Does your app allow for external controls or iOS/Android’s sensitivity options to be used in a beneficial way?

Limited Cognition
Is all text provided straightforward? Is it clear what buttons and actions will do? Is too much information provided at one time? Is there a clear Help menu?

Certainly, not every app can be optimized for every form of disability – it’s expected that a karaoke app wouldn’t be optimized for someone with limited vocal capabilities, and that a music app wouldn’t be optimized for someone with a hearing impairment. But for many game and app developers, accessibility testing is an opportunity to make your product more inclusive and extend its appeal to a much wider audience.

To learn more about VMC’s Accessibility Testing, as well as our other mobile QA services, contact us.

Mobile QA with Accessibility in Mind

Two Trends Impacting the Modern Games QA Team

We asked Zack Hiltz, VMC’s Games QA Services Manager, what the future of Games QA will look like. Here are two trends he’s seeing that will affect the composition of future QA teams. 

Throughout my career in Games QA, I’ve been asked questions like, “Where do you see QA being in five years? What will a Games QA team look like then?” I always did my best to provide an insightful response, knowing full-well that Games is an industry like no other, an endlessly variable playground of design, innovation, and emergent technology. No other major entertainment medium presents the same multitude of means by which consumers access and interact with content. It truly is an industry of infinite possibilities, and with it, infinite challenges.ControllerPic2

Despite this, it’s QA’s role in the development lifecycle to prepare and account for every possible outcome to the best of their abilities. When preparing for this role, it is imperative that you start at the beginning, thinking carefully about the type of team you’re going to build and how it will shift and evolve as the production does. To help with this, here are a couple of trends affecting game QA today and points to consider when putting together your next QA team:

The Blurring Line between DevQA and FQA

I’ve worked with QA teams of vastly varying sizes, from quick in-and-out five-person strike teams to massive 200+ tester operations running year-round. Regardless of size, the same principle always applies: no matter how big your QA team, those man-hours of coverage throughout production will never compare to the endless ways thousands of end-users will tear into your code once it goes live. This is further compounded by the increasing complexity of games software year after year.

To combat this, many QA Managers are recognizing the potential futility of simply ramping headcount ever-upward and are instead actively exploring options to maximize their current QA team’s effectiveness. As a result, black-box functional testers are often being exposed to more advanced testing techniques, combining aspects of white-box methodologies to further their understanding of the game’s internal infrastructure without needing to dive as deep into the code. FQA testers are then able to execute more highly-targeted tests, leveraging their advancing familiarity with the game and its debug commands to explore a broader realm of defect detection. For example, working with parameters to further stress engine stability, or comb through data to identify design issues before a black-box tester might uncover it naturally.

This gray-box hybrid approach is already familiar to many who have worked in development or served as DevQA, yet even in a large-scale QA operation, DevQA headcount tends to be incredibly small (proportionately-speaking) due to the nature of being shoulder-to-shoulder with the development team. Nonetheless, studios are seeing a notable benefit in deploying this more code-relevant testing approach to the wider QA team, naturally leading to an ever-shrinking gap between the direct, high-volume quality verification nature of standard FQA and the more nuanced, isolated expertise of DevQA.

Staffing your QA Team with Your Audience

The video game market today is more inclusive and expansive than ever before. It is no longer in the hands of a select few or dominated by any one particular genre; new developers arrive on the scene every day and new genres are constantly being created or revived from ages past.

The reason is audience. The ever-expanding footprint of the games industry across global cultures has led to an incredibly diverse consumer-base where everything is relevant to someone. If you build it, they will come. Conversely, no single game can dominate an industry this big because no single game can be everything to everyone, and some developers are taking note. Rather than casting the widest-possible net, many are aspiring not for global domination, but to instead carve out their own little corner of the industry and double-down on their key player-bases.

But despite an ever-growing population of gamers year-after-year, one thing remains the same:  gamers can be hard to please. In an increasingly eclectic industry, it is imperative that the people in your little corner are satisfied.

To help with this, QA teams are increasingly staffing their projects with the right individuals for the job, not just in terms of technical expertise, but in terms of high-level game expertise, too. Equipping your QA roster with genre familiarity is nothing new, but in the modern gaming market, meeting the increasing quality demands necessitates staffing beyond simple familiarity and doing your best to source expert-level individuals. These Subject Matter Experts (SME) help guide and execute the testing from a perspective that is indicative of both a game’s general target audience as well as the highly passionate subset of fans that tend to be the most vocally active in the community. SMEs can prepare for and catch the hardest-to-find issues because they themselves are among those passionate fans.

This practice is common in franchise development, for example bringing in expert-level fans of an original entry of an IP to test the sequel. But one area where we’re seeing an increase in this practice is the live gaming environment. As online games grow longer-and-longer legs post-release due to unending content cycles and the revenue models to sustain them, the player base in turn becomes increasingly accustomed to the game, its mechanics, and its possibilities. In order to retain these customers, developers need to not only satisfy a constant stream of updates, but also maintain a certain bar of quality that speaks directly to them.  To help accommodate this, QA teams working on live games tend to be staffed at a higher ratio of SME to non-SME than other projects. This ensures that any new patches or content updates are exposed to and filtered through as much subject matter expertise as possible within a much smaller development cycle.

This approach has proven invaluable in the modern games space, and I expect the demand to grow as developers continue breaking new ground, building new experiences, and seeking new audiences.

VMC’s games expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com/games

Two Trends Impacting the Modern Games QA Team

It Takes a Team: a Conversation with VMC President Juliana Su

Since becoming VMC President in 2016, Juliana Su has had a powerful impact on both the company’s services and its culture. We asked what brought her to VMC, and where she wants to lead the company in the future. Here’s part one of our Q&A.

What excited you to join VMC?

My career has always been with technology companies, from start-ups to large publicly traded companies. I headed up and grew global Professional Services departments for two start-ups and eventually became Chief Operating Officer for one. I wanted a short break from the demands of the corporate world and started by own consulting company, and that’s when VMC came along. It was an opportunity too attractive to pass up – that is, the chance to work with an established technology services company with a solid reputation in the Games industry. Upon joining VMC, I quickly learned that VMC has a lot of talented people, an impressive customer base, and strong services offerings. The company just needed some leaders with technology and Games experience who could quickly operationally re-focus VMC to function like an agile, technology services company.

Describe the culture at the company and your vision for VMC.

I’ve always found that technology does not age well – that is, something new is always on the heels of a current trend. VMC’s biggest asset is its people – people who can think out of the box, who are agile and collaborative and see opportunities where they are not obvious. I’ve always said that I can’t do this alone – it’s not my vision we’re implementing but one that we all collectively believe in. I’ve been with and advised enough start-ups to know that one person alone can’t single-handedly make a company successful. It takes a team. We are all here to take on a role and do it well. That’s what makes VMC successful.

I also don’t believe in a “walled” executive team – everyone at every level should be approachable, accessible, and just one phone call or email away. VMC started 16 years ago with a few opportunities and some individuals with initiative, courage, and moxie. I want us to get back to those roots – let’s be brave and go after opportunities, knowing that sometimes we win and sometimes we lose. Let’s work smartly and embrace our mission – “to ensure the most innovative companies in the world deliver an excellent product experience to every customer everywhere” – and quickly evolve and adjust along the way.

What is the company doing to support and expand that culture?

VMC is headquartered in Redmond, WA, home to some of the most successful technology companies in the world, several of which are our customers. During one leadership meeting, we were discussing how challenging it can be for companies, even established ones such as VMC, to navigate the push toward overall growth while continuing to drive hard toward operational and financial efficiencies. We discussed how this becomes even more daunting for start-up companies and how VMC is well-positioned to help since we’re going through a transformation ourselves. Fortunately, we have talented employees and an experienced leadership team. We decided we needed two paths – one external and one internal. That’s where the ideas of our EXP Lab incubator and Ideation Forum were born.

We decided to tap personal and business connections in the Games space to get the word out about the VMC incubator, an “if we build it, they will come” mentality. VMC has available lab space in the Redmond office as well as strong team members who understand the Games industry well. Our BD team leveraged VMC’s attendance at Games conferences to identify candidate companies for the incubator. And thanks to the hard work of many, we’ve recently signed on our first start-up and are evaluating another.

HR departments will tell you that employee engagement and satisfaction is about more than just compensation. It’s about liking and resonating with the people you work with, seeing opportunities for career advancement, having a voice and influence in the direction of your company, having a sense of ownership, and having a strong corporate culture.

At my previous company, we had an Ideation Forum where employees could submit ideas and receive recognition and a small award for the one voted most innovative. We decided to take it one step further at VMC. A committee comprised of all VMC departments (and no LT members) would vote and select the most innovative idea at the beginning of FY Q4, just in time for the budget planning for the next FY. The idea owner would be responsible for drafting a business case (with support, of course) and meeting with VMC’s CFO to allocate FY budget for implementation. Throughout the fiscal year, the idea owner is responsible for project managing the idea from inception to implementation. All with the support and collaboration of the VMC Leadership Team.

And last, but definitely not least, are internship positions for “up and coming” Games talent. VMC connected with some of the local Games Technology Institutes earlier this year. The original purpose for these connections was to educate ourselves on what’s new and what’s coming. We thought “what better place to do this than where talent is educated and trained.” After a few meetings with the heads of these schools, we discovered that in addition to getting an inside track on the latest trends, we could help broaden the experience of students through internships at VMC or with our customers, thereby investing in our future. It’s still a work in progress but we’re really excited about the possibilities.

Since you’ve been in the consulting space for over a decade, especially in mobility, how do you set the direction for VMC leveraging its core business?

To start, I would have never said that I was a “gamer” before I joined VMC. I have apps on my tablet and phone that are strategy puzzles, and I’d occasionally play Minecraft with my daughter. Then I started working at VMC and realized the breadth of Games is well beyond the console and video. There is a fast-growing marketplace for gaming on mobile devices and in the Cloud.

The reach of mobility is ubiquitous and vast. According to a recent GSMA report, there are 5B mobile phone users worldwide today and in the US alone, 77% of those are smartphones. The mobility addressable marketplace is huge with respect to anything that touches the consumer. And in terms of the Gaming industry, this includes everything from traditional video gaming to VR/AR gaming to casual gaming to loyalty programs.

What does this mean for VMC? That we continue to invest and grow with our clients so they can better serve their customer base. We have a long and strong history of delivering quality services to Gaming clients. To remain relevant, we need to stay ahead of the growth curve, not only continuing to deliver those quality services but evolving to include thought leadership around how to enhance the user experience in the Games space. It’s no longer just about the number of defects that are found and corrected, but about gathering and, more importantly, interpreting data about how Games are being played, player’s preferences, and predictive trends. Our investment will include Technology and Games-specific data analysts who can interpret data and provide predictive business intelligence on how to enhance the player experience, the trends that will lead to the next “big” game, how to keep a loyal fan base, and more.

In parallel, we continue to chase opportunities in the connected consumer space, including all types of connect devices — ones that are worn, driven, and inhabited. By leveraging our Gaming industry expertise in QA and Compatibility Testing, Certification and Localization, we can pivot to offer these relevant and much sought-after services in the connected marketplace as well. This diversification of our client portfolio will generate the revenue and income that will allow VMC to continue investing and innovating in our core businesses: Games and Customer Care.

Watch for part two of our Q&A with Juliana Su soon.

It Takes a Team: a Conversation with VMC President Juliana Su

VMC Welcomes Triangul8 to Its EXP Lab Incubator

VMC is pleased to introduce Triangul8 as the first partner for VMC’s new games incubator, EXP Lab.

EXP Lab is an innovative new incubation program that gives indie game developers their own dedicated lab space and secure connectivity inside VMC’s Redmond office. Moreover, the program enables developers to learn from industry experts who have worked on numerous indie, mid-sized, and AAA games, including attending informational sessions aimed to help Indies prepare for the specific challenges most developers face:

  • Mentorship in QA, Localization, Development, and Production cycles
  • Technical consultancy to meet first-party requirements on all platforms
  • Financial, Legal, and Marketing advice

Triangul8 is the maddeningly addictive triangle strategy game for iOS and Android. Designer James Thomas Charles Hutt said, “We are excited to partner with VMC to teach the world how to Triangul8. Our game is a ‘chess killer’ for the modern day, designed to be easier to learn and play while also activating the same strategic centers of the brain with a more streamlined two-player experience.”

“We’re thrilled to be working with Triangul8 here in Redmond,” said Amy Nanto, Head of Business Development for Games at VMC. “VMC has a long history of supporting Indies, many of which have become industry leaders. EXP Lab continues our ongoing effort to give indie developers the right tools and support to grow. We’re looking forward to seeing Triangul8 thrive.”

“Every game starts out as a fun idea, but there’s a lot that needs to happen to turn that passion into a successful product,” says VMC President Juliana Su. “With over 15 years of experience helping the top game publishers succeed, we are now committed to providing that same expertise to help indie developers get started.”

To learn more about EXP Lab, contact VMC.

Learn how to play Triangul8 at www.triangul8.com.

VMC Welcomes Triangul8 to Its EXP Lab Incubator

How and Why Developers Should Code for UI Automation

VMC’s SDET3/DEV 1 Chris Stephenson looks at when and why to use element IDs, the usefulness of events for automation, test content, and other insights to help get your UI automation running smoothly and accurately. 

Yep, it happened: your manager just announced that all projects will now have automated UI testing. Why?! It’s been tried. It never gets us anywhere. After spending hours running the tests, more often than not, they’re broken, which means more hours trying to figure out what broke the tests. When they aren’t broken, they’re reporting “bugs” that aren’t bugs, just out-of-date tests. And then there are theTestAutomationIcon bugs that DO get through the UI automation.

The benefits from automation are well known for things like networking and API calls. What if we could get some of the same benefits in our UI test suite? It can be done, but not without some work. Let’s take a look at the UI automation pain points and see what developers can do to help fix them.

“I’m Going to Need to See Some ID”

UI automation is slow. To some degree, that is inherent in the process. Unlike a unit test or API call test, the UI automation needs to bring up the UI, with all of the underlying objects. This takes time and memory. Once this is done, the tests can actually start performing their assigned actions. In most cases, that’s something along the lines of scanning through all of the elements of the UI under test, then doing a series of comparisons against various properties to find the element that needs to be interacted with. Frequently, that goes something like this:

“Go find the 2nd element that is a container, then look inside that for an element that has ‘Eat at Joes’ in the title.”

So now the automation has to scan all the elements by type looking for containers, then dig into the children of the second container and read the titles of all of those elements until it finds ‘Eat at Joes’. Alternately, the test could do this:

“Go find the element with ID ‘JoesSign1’.”

Now the automation scrubs through just the IDs of all the elements, which are intentionally made readily available for accessibility coding, and returns the element with the matching ID. In most cases, this is a couple orders of magnitude faster.

By simply editing the automation ID to something unique, a great deal of time can be saved. But that’s not all. Let’s look at that first test command again:

“Go find the 2nd element…”

The automation scans through the object model from top to bottom, root to leaf. We’re adding a new feature to the product that is going to put another container-type element between the 1st and 2nd container elements. What just happened to our test that is running that command? Yeah, busted. So again, simply updating an existing field to something unique will go a long way toward speeding up and stabilizing UI automation. If you only do this one thing, and stop reading here, you will have done wonders for your UI automation. But wait, there’s more.

“The Waiting Is the Hardest Part”

I can’t begin to tell you how many tests I’ve refactored, or worse, written, with something like this in the code:

“Press the big red button, then wait for…oh, I don’t know, 30 seconds?.. then check to see what happened.”

Really? You hit the button to make it do something. You know it’s going to do something, that’s why a button was there. Why are you waiting for a set time? Just listen for the something to be done. Oh. Right. There is no OnButtonDone event to listen to. This is the root cause of the “UI automation takes forever to run” complaint. Nothing slows automation down as consistently or as heavily as Thread.Wait();. As a developer, you made that button. It does a thing. You know what that thing is. You know when it’s done. So add a simple OnButtonDone event, and fire it when your button is done doing its thing.

I hear you. “Just reduce the wait time – problem solved”. Sure, 30 seconds seems excessive. Unless your button is going to query a database that frequently has a lot of work to slog through, and the database is set to a 30 second timeout. If I don’t wait at least as long as the timeout, I run the risk of calling the test a failure even though the data came back as valid, if a little slow. But what about the days when the database isn’t busy? When the data returns in 30ms? How much time saved, test over test, by being able to definitively know when it is safe to check the return data? The results are generally painful, all because of a simple OnButtonDone event.

“What, Exactly, Are We Looking at Here?”

Many products are designed to provide users with a way to interact with some kind of dynamic content. As such, the products tend to use the content as a support and driver in UI development and real-time behavior. Unfortunately, automated tests are far more difficult to design in a similarly adaptable manner because they need to be able to definitively say “yes, that worked” or “nope, it’s busted.” Changes in content dramatically increase the complexity, and therefore stability and maintainability, of UI automation.

The best way to get around this particular problem is to provide test content. You already have a mechanism to swap content easily (that’s the point of your product), so use that feature to give the tests a little love. Build out test content and provide an access method to the tests that can be used to replace the live content. Alternately, provide your test engineer with a mechanism for inserting their own content at test run-time.

To be clear, this is for testing around truly dynamic content that changes without alteration of the product code, and can potentially change the way the product code behaves. Anything with a data-driven catalog of “things” that are altered outside of the product and subsequently displayed or interacted with by the product user (any inventory system, ever) would qualify. If the “dynamic content” is feature changes, or a set number of scripted states that can’t be altered without rebuilding the product, then this doesn’t really apply.

“So, When You Say ‘No’, What You Really Mean…”

You’re building an awesome product, and you don’t want an ugly error message detracting from that when some underlying mechanism glitches, so they get filed away in some dusty directory somewhere, assuming that they get filed away at all. Those glitches are what tests are looking for.

I agree that smacking the user with an error isn’t the preferred course, but test automation needs this data. Specifically, what happened, when, and why, in real-time. Looping back to the OnButtonDone concept, throw in a little bit of data for the tests to catch. A simple success/fail bool is a good start. An enum that can be referenced at run-time to determine the “why” of the failure is even better.

If I’m writing a test that opens a UI, pushes a button, then reads the resulting dialog, I’ll be able to file a much cleaner bug if my test tells me “I hit the button, and it failed because the DB is not responding” instead of “I ran the test, and the dialog wasn’t there.” My test doesn’t know about a database. It doesn’t care. It’s just telling me what the button told it. This way, no one needs to spend hours trying to figure out why the dialog wasn’t there – it was called out in the test.

“Are You Talking to Me?”

I certainly hope so. Clear and constant communication is the best defense against UI automation problems. When a developer and a tester work closely together on a product, automated tests turn out faster, more robust, and more accurate. A tester can’t really tune their automation until the product feature is checked in and functional. They can pre-build and mock out feature behavior, however, and will be far closer to accurate with direct guidance from the developer that is working on the feature. This minimizes the gap between feature complete and automation complete.

Constant communication also clears out a lot of the “false positives” test automation tends to be known for. If the automation is expecting the result of the button push to be a confirmation dialog, and that dialog was removed to minimize user clicks, well, the test will fail on what is ultimately a “good” behavior. It may be that “everyone knew” that the feature was changing, but it doesn’t hurt anything, and helps a great deal, if you touch base with the test engineer anyway.

I hope this helps to clear some things up, and perhaps explains why your test engineers are frequently muttering under their breath about IDs, events, and content.

How and Why Developers Should Code for UI Automation

Adapting a QA Lab for Virtual Reality

 

VR is changing the QA process with its unique requirements. We asked David Kilgour, VMC’s Global Platforms Test Manager, to tell us some of the factors VMC took into account when expanding its VR test facility.

Since VR and AR began showing commercial viability, there have been parallel discussions about both the potential and challenges of the technology. Developers have steadily improved the virtual experience, yet factors such as headset prices, motion sickness, and limited application outside of gaming have been obstacles to widespread adoption. Now, as the industry focuses on perfecting the visual and auditory experience, the user base has grown exponentially, and this is driving more industries to explore VR’s potential, including entertainment, education, online shopping, fitness, and eager marketers in every industry.

QA will be a critical factor in the growth of both VR and AR, and while the goals are the same – get the highest-quality product to market as quickly as possible – there are several key differences between VR testing and traditional console, PC, and mobile-based QA. Some are obvious, others are subtle, but they were all important considerations as we expanded our VR test environment.

Space

While traditional QA can be squeezed into nearly any available space, VR testing needs a lot of room. Testers will be moving around with no sense of the physical space around them. What might be simple for a regular tester to notice, a VR tester may not. This extends beyond the configuration of the furniture and into more daily tasks: if IT is working on an issue for the adjacent tester, the extra people and items are easy for traditional testers to notice but become a hazard for VR testers who, even if they know the extra items are there, may lose track when they’re deep in testing. To maintain a safe working environment, everyone has a responsibility to ensure that the environment remains clear during testing.

Sunlight

I’m not saying that all VR testers are vampires – but imagine for a moment they are. In a more traditional environment, window blinds and lighting would be adjusted throughout the day so that testers have sufficient light (artificial and natural) without direct sunlight on their monitors. VR testers can be immersed in their testing environment for hours without any sense of changes to their real environment, so they could remove their headset to face harsh, direct sunlight. This wouldn’t be good for a vampire or for the VR tester. With this in mind, we plan the space so that the team gets the right amount of light without fear of direct sunlight.

Motion sickness

VR motion sickness is a real thing, and because some people experience it more than others, screening testers is important. It doesn’t matter how effective the tester may be, if they get sick after wearing the headset for 30 seconds (our record is three seconds) then they’re not suitable for VR. Even for those who are suitable, most people get a little motion sick after testing for an extended time, so we provide ginger-based items (ginger candy, ginger ale, ginger tea, etc.) because it helps combat VR-induced motion sickness. Testers can also take breaks as they feel they need instead of being tied to a set break pattern, with some testers taking a few minutes every hour while others preferring a longer break after multiple hours of testing.

Documentation (Test Plans, Walkthroughs, Text Files, etc.)

VR testers are immersed within their environment, so any documentation is difficult to reference within the game, especially a complex and time-critical walkthrough document. Steps are easily missed if a test plan has very specific and cumbersome testing path, and scrolling text is difficult to check against a text file. The easiest way to adjust for many of these issues is having someone available for the tester to talk to. It could be as simple as having a second tester read the test plan as the VR tester moves through the virtual word, and having the VR tester read aloud the text they see and having a second tester check it against the text file.

Genre

With traditional QA, the genre of a title is less of a priority than a tester’s experience on a particular console, but in the VR environment, genre needs to be considered when selecting the right team. Consider the scariest horror movie you’ve seen – now turn that into a video game and place yourself in the middle of it with your VR headset. While it might be amusing to have someone on the team scream the first time a ghost or monster appears in front of them, it can become quite distracting if it was to happen all the time. Before we assign a test team, we speak with each tester individually and give them an overview of the game. If the genre isn’t suitable, we find testers who are more interested.

Potential downside

With every new technology, there is always a downside. We joke that with VR, it is difficult to tell the difference between a tester who is concentrating on one particular aspect and one who has simply fallen asleep.

We Grow as VR Grows

VMC’s VR testing facility is very different from the area we started with a year ago, and as VR technology evolves and player expectations change with it, we’ll continue to make improvements to both our VR testing methods and our space.

VMC’s expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com

Adapting a QA Lab for Virtual Reality

The Path to Proper Mobile Testing

We asked Christian Norton, VMC’s Mobile Manager, to talk about what to look for in a mobile QA partner. 

One of the trends we’re seeing in the mobile space is both new titles and updates being released with performance or launch issues on specific devices. These issues not only leave customers disappointed, they generate negative buzz by getting poor (if not incendiary) user reviews. With the exponential growth of mobile apps and games development, compatibility and thorough update testing are key elements to preventing these situations.IGDA-mobile

When you’re considering a QA partner to test your mobile title, here are several factors you should look at to ensure a great experience on every targeted device:

Experience
What matters here isn’t years in business, but their specific experience: have they tested mobile products similar to yours so they know common pitfalls to avoid? Are they knowledgeable about mobile devices so they can make recommendations on which the game or app should be tested on? Can they identify gaps in a test plan and offer insights on a more effective approach? Do they have in-depth knowledge and experience on mobile compatibility testing, and can they provide manual tests on a wide range of devices? Understanding their expertise in advance enables you to collaborate on a smart, efficient test plan for ensuring your product delivers what it was designed to.

Inventory
Mobile users seem to fall into two camps: those who always want the newest model, and those who refuse to give up their old devices until they stop working. Your testing partner needs to have both the hottest items on the market and older devices that retain a huge user base. Devices are also handed down from parents to children who may be using apps or playing games on older devices or software versions. Whether you are releasing an update or a new title, your QA partner should be able to analyze the latest device trends to determine the best test strategy for your game or app.

Flexibility
It’s great when there’s time to execute a thorough test plan within your project timeline, but what if you find something unforeseen a few days before release, or worse, your new title or update is released and several users complain about problems downloading your app or playing your game? You need a solution fast, and that requires a QA partner who understands the urgency of the situation and can quickly provide an effective test solution to help isolate the problems on your users’ preferred devices.

Whether you’re an indie developer or an established game publisher, QA is more than an outsourced service – it’s an essential element of your product’s success. Look for a QA partner you can count on in any circumstance.

VMC’s expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com

The Path to Proper Mobile Testing