How To Conduct User Research In The Prototype Stage For Your Game

Whether it’s your game concept or your core game features, prototyping and testing ideas at an early stage can you let you know a lot about the health of your game, and more specifically, if people are actually going to enjoy it.

Ideally, you’ll want to make sure your title is popular the moment it hits the market. Not only is testing in the early stages of development good for getting your game out there, but it can also help you focus your production process on features that will make the most impact on your intended audience.

In this post, you’ll find the steps you need to take when conducting user research during that all-important prototype phase. GameAnalytics included a bunch of different examples of specific test methods (with pros and cons of each) as well as advice for pulling together your very own test plan.

This is a bit of a lengthy post, so feel free to skip to a specific section if you’re looking for something in particular:

• Step 1: Plan out your approach
• Step 2: Make sure you know what you’re testing, and why
• Step 3: Choose the right testing method
• Step 4: Reviewing the results
• Top takeaways (TL;DR)
• Useful links
• Your User Research Checklist (useful guide that you can download)

Step 1: Plan out your approach

Testing is great. It can tell you things about your game that you’re blind to, or just confirm suspicions that may have been playing on your mind. However, before you test anything about your game, you’ll need to take a moment to reflect on the purpose of your prototype stage.

This stage is about figuring out exactly what you need to test in order to improve your game, but also laying out the steps you need to take to move forward. A few questions you’re going to need to answer are:

• What is your ultimate objective?
• What are your plans post-test?
• What resources do you have?
• How much time can you dedicate to test?
• Which features do you need to test?

Now is the time to define your strategy for your user research. The approach you take should be tailored to the data that matters to your game development, whether it’s feedback on game angles, ideas, concepts, gameplay, or, simply just the design.

For example, if I’m creating a hyper-casual title with simple mechanics, it would be foolish to test narrative. A better use of time here would be the games core loop, difficulty or level design.

To help you understand what this requires, GameAnalytics created a useful document with the essential questions and checklists you’ll need when building a prototype test plan. You can download a copy here.

Step 2: Make sure you know what you’re testing, and why

What on earth are you testing?

So you have your approach all together, and you’re ready to take the next step. At this stage, you’re going to have to look at your title and decide exactly on what you’re going to test. Depending on the approach, this could either be a particular in-game feature, aesthetics, the overall UX, or even a monetization strategy.

Knowing exactly what you’re testing will inform the whole structure of the user research. If you’re searching for feedback on your game’s concept, for example, you might want to mock-up an app store screen with descriptions and images to see how new players respond.

Who should you be testing?

Once you know what you’re testing, you’ll have a better idea of who you should be testing it with. This refers to the number of participants you need to gather results from, as well as the demographics and player personas you need to target.

If you want to gain more qualitative feedback on whether a specific kind of player will enjoy your game, playtesting sessions that involve face-to-face interviews might be the best option for you. To do this, you’ll need to be familiar with the type of players your game is targeting, considering things like age, gender, platform, player experience levels and interests.

To give you an idea of what we mean, here’s a few different examples:

• Specific audience: Hard-core gamers who play for 20+ hours a week, aged 20 – 40, lives in London
• Less specific: Women, aged 50+ who are interested in puzzle games
• Open: 10 Beta players

Why are you doing this test?

You should have a pretty good idea what the answer is to this question, but it’s important to write this down. It’s easy to get lost in all of the nitty gritty details when conducting your user research, so this all-important ‘why’ will bring you back to Earth, and keep your vision and goal clear.

A few examples of ‘why’ you’ll be testing could be:

• See if players simply enjoy the core features
• Understand first impressions of the game in the app store
• Find out if levels are too difficult or too easy
• Learn how players interact with the core game

How are you going to test?

There’s an appropriate tool for every job. Once you’ve answered the above questions, you should be in a good place to decide on which testing methods are the best fit for your purpose. For instance, your decision will vary massively depending on whether you’re searching for in-depth qualitative feedback on player experiences, or if you’re searching for more general feedback from a large sample of participants.

GameAnalytics actually listed some of the most popular testing methods below to aid your user research during the prototype stage…

Step 3: Choose the right testing methods

A/B testing

As you may already be aware, A/B testing lets you test two or more things at the same time in a controlled environment. These types of tests normally are an iteration of one aspect, as it’s important to make sure you’re testing apples against apples.

For example, you can test a red icon colour against a green icon colour. But you can’t test a red icon colour against a different type of font – the results of this will just confuse you and lead to inconclusive results.

Online surveys

The use of online market research tools has come a long way in the last decade. There are now several game-specific platforms (like Playtest Cloud, Sekg, Player Research, and more) that are set up to offer you easy survey functionality when it comes to testing elements of your game.

Diary studies

Diary studies are a form of long-term research that takes place over a set duration of time. Typically, users report their activities at regular intervals to create a record that developers can study. It’s up to you how this is structured, but it can include player’s thoughts, frustrations or enjoyability levels over a number of sessions.

Expert reviews

Mobile game experts can offer something unique when it comes to prototype testing. Whether it’s a pro gamer or a professional prototype tester, sometimes you need feedback from someone who has a firm grasp of the medium to inform your development strategy. This should help offer actionable development advice that can help you transform specific features of your game.

Face-to-face interviews

There are a few variations for how this test method works. A basic model is having a group of testers play your game for a set period of time, allowing them test out different features and experience the game as a whole. Following this, an interview can be performed to gather specific feedback related to their experience.

Step 4: Reviewing the results

When reviewing your test results, everything should tie back in to your test objectives as outlined in your test plan. In the end, your interviews, studies or surveys are all designed to improve your overall development process, so make sure you use the results to inform your strategy going forward.

The data you receive should also be reviewed as objectively as possible. If your team are too close to the situation to judge it fairly, then bring in some fresh eyes to help you understand your next course of action. It’s important that you don’t let emotions get in the way. If you receive some negative feedback, you simply need to suck it up and adapt your future approach – there’s no two ways about it.

If your results confirm your initial ideas then hooray! You can now act with the confidence you need to invest additional resources into further developing and refining your game, bringing it that one step closer to the finish line.

Here’s an example…

As we are a strong advisor of ‘show don’t tell’, here’s an example of how you can conduct your user research for your next project. Also, don’t forget to download GameAnalytics’ template, which should help you cut some corners when setting up.

Example: We have an unproven game concept in mind, but we don’t know if players will like it

• Step 1: Plan approach
  • We need to find out if our concept will resonate with potential players. To achieve this, we’ll need to mock-up an app store page, with fake images, text and gameplay. Our resources are limited and budget tight, so this route will make sure we don’t exhaust ourselves in case we need to do more testing.

• Step 2: Answer the 4 ‘What, who, why, how’
  • What: A fake app store page, with images, text and mock gameplay
  • Who: Men aged 20 – 35 who work in London and play mobile games (roughly 300 invitations.)
  • Why: To find out if our game’s concept is interesting enough for players to want to download
  • How: Online survey

• Step 3: Set up
  • Create mock assets for testing
  • Decide and set up online survey questions
  • Identify audience and send out invitation
  • Decide on closing date

• Step 4: Review the results
  • At this stage, we should have a collection of results that will help decide our next steps.

Remember…

Due to the high costs associated with developing and launching new games, identifying failing ideas as fast as possible is crucial. This could save you a ton of time in the long run, as well as make sure you’re on the path to success.

When figuring out your approach, don’t get stuck within a rigid test format. What might work for one studio might not necessarily work for you, so tailor your user research to suit your target audience, genre and the development stage of your game (as well as the resources that you have).

Finally, use a collection of different test methods for more reliable results (if you can). Any areas that overlap between different methods will give you more solid conclusions, rather than relying on the success of one method alone.

More on game testing

For more information and advice on game testing, GameAnalytics collected some great links that can help you while you’re conducting research during the prototype phase:

• How To Do An Efficient Game Test
• A/B testing with Custom Dimensions
• UX Walkthrough: Anatomy of a Usability Test in Video Games (Part-1)