VMC Welcomes Triangul8 to Its EXP Lab Incubator

VMC is pleased to introduce Triangul8 as the first partner for VMC’s new games incubator, EXP Lab.

EXP Lab is an innovative new incubation program that gives indie game developers their own dedicated lab space and secure connectivity inside VMC’s Redmond office. Moreover, the program enables developers to learn from industry experts who have worked on numerous indie, mid-sized, and AAA games, including attending informational sessions aimed to help Indies prepare for the specific challenges most developers face:

  • Mentorship in QA, Localization, Development, and Production cycles
  • Technical consultancy to meet first-party requirements on all platforms
  • Financial, Legal, and Marketing advice

Triangul8 is the maddeningly addictive triangle strategy game for iOS and Android. Designer James Thomas Charles Hutt said, “We are excited to partner with VMC to teach the world how to Triangul8. Our game is a ‘chess killer’ for the modern day, designed to be easier to learn and play while also activating the same strategic centers of the brain with a more streamlined two-player experience.”

“We’re thrilled to be working with Triangul8 here in Redmond,” said Amy Nanto, Head of Business Development for Games at VMC. “VMC has a long history of supporting Indies, many of which have become industry leaders. EXP Lab continues our ongoing effort to give indie developers the right tools and support to grow. We’re looking forward to seeing Triangul8 thrive.”

“Every game starts out as a fun idea, but there’s a lot that needs to happen to turn that passion into a successful product,” says VMC President Juliana Su. “With over 15 years of experience helping the top game publishers succeed, we are now committed to providing that same expertise to help indie developers get started.”

To learn more about EXP Lab, contact VMC.

Learn how to play Triangul8 at www.triangul8.com.

VMC Welcomes Triangul8 to Its EXP Lab Incubator

How and Why Developers Should Code for UI Automation

VMC’s SDET3/DEV 1 Chris Stephenson looks at when and why to use element IDs, the usefulness of events for automation, test content, and other insights to help get your UI automation running smoothly and accurately. 

Yep, it happened: your manager just announced that all projects will now have automated UI testing. Why?! It’s been tried. It never gets us anywhere. After spending hours running the tests, more often than not, they’re broken, which means more hours trying to figure out what broke the tests. When they aren’t broken, they’re reporting “bugs” that aren’t bugs, just out-of-date tests. And then there are theTestAutomationIcon bugs that DO get through the UI automation.

The benefits from automation are well known for things like networking and API calls. What if we could get some of the same benefits in our UI test suite? It can be done, but not without some work. Let’s take a look at the UI automation pain points and see what developers can do to help fix them.

“I’m Going to Need to See Some ID”

UI automation is slow. To some degree, that is inherent in the process. Unlike a unit test or API call test, the UI automation needs to bring up the UI, with all of the underlying objects. This takes time and memory. Once this is done, the tests can actually start performing their assigned actions. In most cases, that’s something along the lines of scanning through all of the elements of the UI under test, then doing a series of comparisons against various properties to find the element that needs to be interacted with. Frequently, that goes something like this:

“Go find the 2nd element that is a container, then look inside that for an element that has ‘Eat at Joes’ in the title.”

So now the automation has to scan all the elements by type looking for containers, then dig into the children of the second container and read the titles of all of those elements until it finds ‘Eat at Joes’. Alternately, the test could do this:

“Go find the element with ID ‘JoesSign1’.”

Now the automation scrubs through just the IDs of all the elements, which are intentionally made readily available for accessibility coding, and returns the element with the matching ID. In most cases, this is a couple orders of magnitude faster.

By simply editing the automation ID to something unique, a great deal of time can be saved. But that’s not all. Let’s look at that first test command again:

“Go find the 2nd element…”

The automation scans through the object model from top to bottom, root to leaf. We’re adding a new feature to the product that is going to put another container-type element between the 1st and 2nd container elements. What just happened to our test that is running that command? Yeah, busted. So again, simply updating an existing field to something unique will go a long way toward speeding up and stabilizing UI automation. If you only do this one thing, and stop reading here, you will have done wonders for your UI automation. But wait, there’s more.

“The Waiting Is the Hardest Part”

I can’t begin to tell you how many tests I’ve refactored, or worse, written, with something like this in the code:

“Press the big red button, then wait for…oh, I don’t know, 30 seconds?.. then check to see what happened.”

Really? You hit the button to make it do something. You know it’s going to do something, that’s why a button was there. Why are you waiting for a set time? Just listen for the something to be done. Oh. Right. There is no OnButtonDone event to listen to. This is the root cause of the “UI automation takes forever to run” complaint. Nothing slows automation down as consistently or as heavily as Thread.Wait();. As a developer, you made that button. It does a thing. You know what that thing is. You know when it’s done. So add a simple OnButtonDone event, and fire it when your button is done doing its thing.

I hear you. “Just reduce the wait time – problem solved”. Sure, 30 seconds seems excessive. Unless your button is going to query a database that frequently has a lot of work to slog through, and the database is set to a 30 second timeout. If I don’t wait at least as long as the timeout, I run the risk of calling the test a failure even though the data came back as valid, if a little slow. But what about the days when the database isn’t busy? When the data returns in 30ms? How much time saved, test over test, by being able to definitively know when it is safe to check the return data? The results are generally painful, all because of a simple OnButtonDone event.

“What, Exactly, Are We Looking at Here?”

Many products are designed to provide users with a way to interact with some kind of dynamic content. As such, the products tend to use the content as a support and driver in UI development and real-time behavior. Unfortunately, automated tests are far more difficult to design in a similarly adaptable manner because they need to be able to definitively say “yes, that worked” or “nope, it’s busted.” Changes in content dramatically increase the complexity, and therefore stability and maintainability, of UI automation.

The best way to get around this particular problem is to provide test content. You already have a mechanism to swap content easily (that’s the point of your product), so use that feature to give the tests a little love. Build out test content and provide an access method to the tests that can be used to replace the live content. Alternately, provide your test engineer with a mechanism for inserting their own content at test run-time.

To be clear, this is for testing around truly dynamic content that changes without alteration of the product code, and can potentially change the way the product code behaves. Anything with a data-driven catalog of “things” that are altered outside of the product and subsequently displayed or interacted with by the product user (any inventory system, ever) would qualify. If the “dynamic content” is feature changes, or a set number of scripted states that can’t be altered without rebuilding the product, then this doesn’t really apply.

“So, When You Say ‘No’, What You Really Mean…”

You’re building an awesome product, and you don’t want an ugly error message detracting from that when some underlying mechanism glitches, so they get filed away in some dusty directory somewhere, assuming that they get filed away at all. Those glitches are what tests are looking for.

I agree that smacking the user with an error isn’t the preferred course, but test automation needs this data. Specifically, what happened, when, and why, in real-time. Looping back to the OnButtonDone concept, throw in a little bit of data for the tests to catch. A simple success/fail bool is a good start. An enum that can be referenced at run-time to determine the “why” of the failure is even better.

If I’m writing a test that opens a UI, pushes a button, then reads the resulting dialog, I’ll be able to file a much cleaner bug if my test tells me “I hit the button, and it failed because the DB is not responding” instead of “I ran the test, and the dialog wasn’t there.” My test doesn’t know about a database. It doesn’t care. It’s just telling me what the button told it. This way, no one needs to spend hours trying to figure out why the dialog wasn’t there – it was called out in the test.

“Are You Talking to Me?”

I certainly hope so. Clear and constant communication is the best defense against UI automation problems. When a developer and a tester work closely together on a product, automated tests turn out faster, more robust, and more accurate. A tester can’t really tune their automation until the product feature is checked in and functional. They can pre-build and mock out feature behavior, however, and will be far closer to accurate with direct guidance from the developer that is working on the feature. This minimizes the gap between feature complete and automation complete.

Constant communication also clears out a lot of the “false positives” test automation tends to be known for. If the automation is expecting the result of the button push to be a confirmation dialog, and that dialog was removed to minimize user clicks, well, the test will fail on what is ultimately a “good” behavior. It may be that “everyone knew” that the feature was changing, but it doesn’t hurt anything, and helps a great deal, if you touch base with the test engineer anyway.

I hope this helps to clear some things up, and perhaps explains why your test engineers are frequently muttering under their breath about IDs, events, and content.

How and Why Developers Should Code for UI Automation

Five QA Tips for Indies

Marc-Andre Legault, VMC’s Test Manager for games, shares his insights on maximizing the impact of a limited QA budget.

With the rise of affordable technology and a new generation of creators working on games starting in their teens, releasing an indie game is within everyone’s reach. Unfortunately, the side effect of this incredible boom is that most indie titles struggle to stand out and sell in an overly crowded market.

In the days of Bastion, Hotline Miami, and Super Meat Boy, there was a perception that every new indie title was stronger than the last and polished to perfection. Now, with thousands of titles being released, many in a less than optimal state, some see the “indie” tag as synonymous with unpolished and unfinished games.

Impressive graphics and frames per second don’t guarantee a title’s success – it has to deliver an enjoyable experience. Consider the case with Minecraft: part of what made Minecraft an early success was that the ambitious initial release, while nowhere near what it would become, was reliably functional. No matter how good a title’s ideas are, if the functionality isn’t working as intended or the player can’t get through an area without having to deal with severe bugs, it won’t survive in a market where many titles live or die by their initial release. Even recent AAA titles have struggled to come back from their initial release states even after multiple patches were released.

This is where a good QA partner can help, ensuring that an indie release offers a satisfying experience, unencumbered by bugs. Since indie titles are usually self-financed and every dollar matters, how does a small team successfully deliver high quality while still controlling costs? The answer lies in how to work with your QA partner, targeting when and where they use their QA resources.

Here are five factors to consider for improving your QA process and maximizing its value:

  • Start at the right time: don’t start so early that many of the launch features are not implemented, and not so late that some UX concerns cannot be reported and addressed.
  • Strategic staffing: once QA starts, try to keep at least one tester on the project (assuming the project is Single Player only) for the duration for the more granular coverage, and bring in a bigger team on key development milestones.
  • Create a good debug tool: limited QA time means making the most of time available, and having the right debug commands available can make a difference in reaching your goals.
  • Create and maintain a Game Design Document: regular updates and/or detailed build notes reduce the chance of invalid issues being reported. While this is recommended for any size project, it’s crucial for a small team/budget.
  • Keep a portion of your budget for post-release: one side effect of having a smaller QA team is that once the title is deployed to the general public, a higher number of out-of-path issues will be found and will need addressing.

By keeping these factors in mind, an indie team can have a good portion of the QA groundwork laid out. In an industry where indie titles are forced to fight for space in the market, good QA can be a real differentiator.

VMC’s expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com

Five QA Tips for Indies

VMC’s EXP Lab Helps Indie Developers Grow

Independent developers are among the most creative talents in the industry, surprising audiences with fun and engaging new IPs. Many indies have a strong vision for their game or app, but they lack access to the array of knowledge required to transform their brilliant idea into a successful product.

VMC’s EXP Lab is an innovative new incubation program that gives indie developers access to a secured lab environment and information, technology, and mentoring that will enable them to focus on making great games. This program enables developers to learn from industry experts who have worked on numerous indie, mid-sized, and AAA games, including attending informational sessions aimed to help indies prepare for success.

Along with space and infrastructure in a secured lab environment at our headquarters in Redmond, EXP Lab participants will benefit from:

  • Mentorship in QA, development, and production cycles
  • Technical consultancy to meet first-party requirements on all platforms
  • Financial, Legal, PR, and Marketing expertise
  • Local community in games

Even the most talented games professionals can face game development challenges: minimal experience with thorough QA/localization testing, limited understanding of platform certification requirements, limited budgets and access to hardware, and/or insufficient information on best practices for start-to-finish development. EXP Lab is designed to fill these gaps and provide guidance for addressing a wide variety of business challenges.

VMC’s mission is to ensure the most innovative companies in the world deliver an excellent product experience to every customer, everywhere – and many of the most innovative ideas are coming from indie developers. We have a long history of supporting indies, many of which have become industry leaders, and EXP Lab continues this ongoing effort to give indie developers the tools and knowledge to grow.

VMC’s expertise ensures ​the most innovative companies ​in the world deliver an excellent ​product experience to ​every customer, everywhere. Learn more at vmc.com

VMC’s EXP Lab Helps Indie Developers Grow

Why indie developers need to think about more than development

Kirstin Whittle (Sr. Manager, Business Development) shares her thoughts on why creativity alone isn’t enough to ensure success. 

I had the pleasure of attending the 2015 Nordic Game Conference in Malmö and was impressed with so many creative, ambitious new games.

Indie games continue to thrive in Scandinavia and Europe, and I enjoyed talking with many start-up developers along with a number of industry veterans working on new ventures, to learn about what they’re doing and where they want their games to go.

The indie arena is great for developers who want more freedom and control to create the games they want to play, and the ongoing evolution of publishing models makes it possible to develop and release games that likely never would be released by an established publisher that expects a certain return on their investment.

But as is the case for any game, creativity isn’t enough to spur a game’s success.

Prepping for success

When the primary focus is on development, global production support services may not be a key focus, but developers need to have excellence in every area of their business, including marketing, legal, QA, localisation, live game operations and community management.

Managing quality in all of these areas is essential to getting people to play and keep playing your game because a great idea won’t make a splash if it’s poorly tested or localised, or isn’t marketed to your target audience.

No one knows what the next big thing is going to be, and many indie developers aren’t prepared to scale up quickly when their game takes off. Global production support services need to be an integral part of a development plan, but there’s no one-size-fits-all solution for any game.

Here are a few key factors to consider:

Pick a partner, not a provider

A good partner will serve as a guardian of your IP, and will understand that your game is your baby. Many global production support companies will take on indie projects and even offer advice prior to a formal engagement because they also want to be a part of the next big thing.

Talk to support partners who you trust and who have expertise to fill the gaps in the areas your in-house team is lacking.

Get them involved early

Getting your support partners involved early enables developers to benefit
from the insights an experienced partner can provide.

This includes anticipating obstacles and pitfalls, planning for all contingencies, and having additional support specialists who are already in the loop, who know your game, and with whom you have an established rapport.

Think beyond the launch

You want to offer more than just a great game – you want to give your customers a great experience, and this may include live game operations (games-as-a-service), customer support, and various forms of community management.

While an indie game may have humble beginnings, having it become a success doesn’t mean your relationship with your global production support partners has to change.

Many developers see augmenting their staff to bring everything in-house as a sign of success, but the peaks and valleys of the production process mean the workload for different groups will ebb and flow.

Continuing to work with outsourced partners can remain the most cost-effective approach, especially partners who already know your game, have proven their value, and can continue to provide critical subject matter expertise while you maintain complete creative control of your IP.

So pick good partners and get them involved early and you’ll be ready for anything.

Why indie developers need to think about more than development