Playtesting
How to run a playtest, and then collect / analyze data
A playtest is when someone plays your level, and you watch whether your level works or not.
Playtesting is a foundational skill and process in game design. You should do it.
Playtests are when you witness whether your level is "fun" or boring, clear or confusing, pleasant or annoying. If players hate your level, you should deal with that problem as soon as possible. "Playtest early and often."
Level design takes a long time, and it is natural to lose motivation during a project. Playtesting with someone else can boost your energy when you see someone else enjoying your terrible broken half-broken map, despite your misgivings.
The commercial game industry takes playtesting so seriously that they perform many phases of testing like user research, QA, test markets, usability, certification, etc. The biggest game companies even run their own dedicated playtesting labs.
a photo of an industrial-grade user research usability lab by Experience Dynamics; game testing labs are similar (but uglier)
Each playtest has a different purpose, depending on the audience.
- Self-playtest: you test your own level.
- You should playtest your level yourself every day, as part of the design process
- Good for catching obvious problems (does the level load? is the doorway wide enough?)
- Critique: someone else (usually a fellow game dev) playtests your level and gives you knowledgeable feedback afterward. You can even have a conversation to brainstorm how to solve the problems.
- Ideally, do this after every phase: after layout, blockout, scripting, and art pass, etc.
- This is where a lot of learning and collaboration happens, very common in workplace
- Good for getting help on "wicked problems" -- problems that are so complex that you can't even describe the problem, much less solve it
- Public playtest: someone else (ideally a "normal person" / member of the public, and not a developer) playtests the level.
- Watching their behavior is more important than their feedback
- Very important for catching hidden problems ("unknown unknowns")
- But normies have a hard time playtesting blockouts, you'll likely have to art pass first
- "Blind" playtest: a public playtest but with minimal guidance or prompting; pretend they are playing the level by themselves, alone, and see what they do
photo of various people playtesting games at a "Playtest Thursday" event at NYU Game Center circa 2014
- Critiques: join an online level design community, ideally a forum or Discord with specific experience in the genre or game engine.
- If possible, try to do an in-person playtest. See if there's a local International Game Developers Association (IGDA) chapter and/or indie meetup.
- You need people with enough game design experience to contextualize their feedback appropriately -- people who know how to look past unfinished systems, obvious bugs, or placeholder art, and get to the real problems at the heart of the design.
- Public playtests: put your level in front of friends or family members, who are often socially / emotionally obligated to engage with your interests.
There are many ways to run a playtest session.
Below is a general process for an in-person public playtest:
Before you playtest, you should figure out what you're testing! Treat it like a science experiment -- form a hypothesis and design a procedure.
Here's a basic playtest prep checklist:
- Scope: How much of the game / level are you testing? How long will it take?(example: "the first half of level 2, for 15 minutes max")
- Focus: Which parts are most important to test? What are your biggest questions?(e.g. "is the level exit clear enough? is the boss too difficult?")
- Briefing: What instructions will you give to the playtester? Any content warnings?(e.g. "use WASD to move, but don't press Space, I haven't fixed the jump yet... oh and the game has spiders, are you OK with spiders?")
- Data: What data are you collecting? How will you collect it?(e.g. use stopwatch app to time how long it takes for them to reach boss #1)
- Notes: Are you taking notes / recording observations? How, where?(e.g. write notes in notebook with a pen... oh no I forgot to bring a pen!...)
Briefly introduce the concept and how long the playtest session will run, set some basic expectations to respect the playtester's time and energy. If the playtest session may contain jump scares, spiders or insects, flashing or flickering lights, sexual content, or graphic/explicit violence, then give a content warning and get the player's informed consent.
You can also tell the playtester what you want feedback on, and set some basic boundaries and focus for the playtest. A more specific briefing is very helpful if your playtester is a fellow designer, so they can tailor their feedback.
Example briefings:
- "use WASD and mouse... have fun"
- "I'm looking for combat ideas, tell me if the encounters are any good"
- "I haven't art passed the level yet so just focus on layout, these are all placeholders"
- "try to use the double jump a lot, I want to see if it works well for this level"
- "the layout is final and we can't change it for budget reasons, so please only give me feedback on enemy placement and pickups"
But keep it brief. Don't over-brief and don't dwell too much on disclaimers, especially if you're having other developers playtest. As designers we often feel insecure or anxious about how our work will be received, but We understand your levels will be unfinished or work-in-progress, with placeholder assets and bugs everywhere.
Most of the time should be spent watching the screen, taking notes about player behavior, reactions, or anything that needs fixing. Make sure to watch from a short distance behind the playtester(s), don’t just breathe over their shoulder.
Feel free to ask brief clarifying questions about the playtester's behavior or thought process, but resist the urge to interrupt the playtest to apologize or explain. Whenever the playtest goes wrong, let that discomfort and anxiety stew in your mind for a little while. However, if the playtester is definitely stuck or if the level is clearly just broken, then go ahead and intervene.
If it is not an in-person playtest, then ask the tester to record their screen and send a private link to the video upload. Or better yet, the playtester can broadcast a private live stream and take questions via chat or voice. For multiplayer maps, it is best to ask an existing player group, server, or clan to play on the map and allow observers.
Track basic stats such as play time, deaths, win percentages, etc. Most of this stat collection does not require any special plugins or tools, just attentive note-taking.
Some games can automatically aggregate player data and visualize patterns, especially for multiplayer maps. Do players use the shotgun too often on a map intended for snipers, or are too many players dying outside the spawn room? A stats server can track players and display the data in a table / spreadsheet. Some level designers even use heatmaps, top-down visualizations of where players frequently move, earn points, score kills, or die.
Ask for feedback and debrief the playtester.
Have a few survey questions ready for the playtester. You could ask comprehension questions ("Did you notice the orange light above the door? What do you think it meant?") or ask for more subjective opinions ("Did the boss fight feel easy or difficult?)
Be kind and respectful to your playtesters even if they're giving you feedback that you don't necessarily want to hear. If someone offers suggestions in good faith, smile and take it with grace no matter what. The suggestion itself might be a poor solution to a design problem, but the reason for the suggestion -- and the issue it highlights -- is almost always real and valid.
But also remember, the playtest session is not for the tester's benefit. If the playtester has a harmful attitude or gives abusive feedback, especially at an in-person playtest session, then you can totally just end the playtest and ask them to stop playing or talking.
Being part of a creative community means taking feedback, as well as delivering feedback, in a respectful and helpful way. The goal of a playtest is not to show-off your gamer skills, the goal is to help the designer predict whether their game works for their audience.
If you're new to game development, you're probably being either way too nice or way too nitpicky. Constructive negative feedback usually helps designers more than positive feedback. At the same time, there's no point in repeatedly breaking the game (unless the dev asks you to) or exaggerating any problems.
- 1.If you are unfamiliar with the game genre or design style, say so.
- 2.Ask the developer, "what do you want feedback on?"
- 3.Ask the developer, "is there anything I should know before I start?"
Last modified 1yr ago