We asked Zack Hiltz, VMC’s Games QA Services Manager, what the future of Games QA will look like. Here are two trends he’s seeing that will affect the composition of future QA teams.
Throughout my career in Games QA, I’ve been asked questions like, “Where do you see QA being in five years? What will a Games QA team look like then?” I always did my best to provide an insightful response, knowing full-well that Games is an industry like no other, an endlessly variable playground of design, innovation, and emergent technology. No other major entertainment medium presents the same multitude of means by which consumers access and interact with content. It truly is an industry of infinite possibilities, and with it, infinite challenges.
Despite this, it’s QA’s role in the development lifecycle to prepare and account for every possible outcome to the best of their abilities. When preparing for this role, it is imperative that you start at the beginning, thinking carefully about the type of team you’re going to build and how it will shift and evolve as the production does. To help with this, here are a couple of trends affecting game QA today and points to consider when putting together your next QA team:
The Blurring Line between DevQA and FQA
I’ve worked with QA teams of vastly varying sizes, from quick in-and-out five-person strike teams to massive 200+ tester operations running year-round. Regardless of size, the same principle always applies: no matter how big your QA team, those man-hours of coverage throughout production will never compare to the endless ways thousands of end-users will tear into your code once it goes live. This is further compounded by the increasing complexity of games software year after year.
To combat this, many QA Managers are recognizing the potential futility of simply ramping headcount ever-upward and are instead actively exploring options to maximize their current QA team’s effectiveness. As a result, black-box functional testers are often being exposed to more advanced testing techniques, combining aspects of white-box methodologies to further their understanding of the game’s internal infrastructure without needing to dive as deep into the code. FQA testers are then able to execute more highly-targeted tests, leveraging their advancing familiarity with the game and its debug commands to explore a broader realm of defect detection. For example, working with parameters to further stress engine stability, or comb through data to identify design issues before a black-box tester might uncover it naturally.
This gray-box hybrid approach is already familiar to many who have worked in development or served as DevQA, yet even in a large-scale QA operation, DevQA headcount tends to be incredibly small (proportionately-speaking) due to the nature of being shoulder-to-shoulder with the development team. Nonetheless, studios are seeing a notable benefit in deploying this more code-relevant testing approach to the wider QA team, naturally leading to an ever-shrinking gap between the direct, high-volume quality verification nature of standard FQA and the more nuanced, isolated expertise of DevQA.
Staffing your QA Team with Your Audience
The video game market today is more inclusive and expansive than ever before. It is no longer in the hands of a select few or dominated by any one particular genre; new developers arrive on the scene every day and new genres are constantly being created or revived from ages past.
The reason is audience. The ever-expanding footprint of the games industry across global cultures has led to an incredibly diverse consumer-base where everything is relevant to someone. If you build it, they will come. Conversely, no single game can dominate an industry this big because no single game can be everything to everyone, and some developers are taking note. Rather than casting the widest-possible net, many are aspiring not for global domination, but to instead carve out their own little corner of the industry and double-down on their key player-bases.
But despite an ever-growing population of gamers year-after-year, one thing remains the same: gamers can be hard to please. In an increasingly eclectic industry, it is imperative that the people in your little corner are satisfied.
To help with this, QA teams are increasingly staffing their projects with the right individuals for the job, not just in terms of technical expertise, but in terms of high-level game expertise, too. Equipping your QA roster with genre familiarity is nothing new, but in the modern gaming market, meeting the increasing quality demands necessitates staffing beyond simple familiarity and doing your best to source expert-level individuals. These Subject Matter Experts (SME) help guide and execute the testing from a perspective that is indicative of both a game’s general target audience as well as the highly passionate subset of fans that tend to be the most vocally active in the community. SMEs can prepare for and catch the hardest-to-find issues because they themselves are among those passionate fans.
This practice is common in franchise development, for example bringing in expert-level fans of an original entry of an IP to test the sequel. But one area where we’re seeing an increase in this practice is the live gaming environment. As online games grow longer-and-longer legs post-release due to unending content cycles and the revenue models to sustain them, the player base in turn becomes increasingly accustomed to the game, its mechanics, and its possibilities. In order to retain these customers, developers need to not only satisfy a constant stream of updates, but also maintain a certain bar of quality that speaks directly to them. To help accommodate this, QA teams working on live games tend to be staffed at a higher ratio of SME to non-SME than other projects. This ensures that any new patches or content updates are exposed to and filtered through as much subject matter expertise as possible within a much smaller development cycle.
This approach has proven invaluable in the modern games space, and I expect the demand to grow as developers continue breaking new ground, building new experiences, and seeking new audiences.
VMC’s games expertise ensures the most innovative companies in the world deliver an excellent product experience to every customer, everywhere. Learn more at vmc.com/games.