So they’ve identified a flaw in your game. But you aren’t sure about how to fix it. What do you do? Well, you can make an educated guess. But the best developers don’t guess.
They know they use A/B testing, which tests different versions of a game simultaneously. And seeing which version gets the best response from players.
It’s called AB because the most straightforward test compared to two versions. An A and B version. The A version is given to one-half of the games users. And the B is given to the other half, and we see how each user group responds. These tests can help you repair any game content you’re unsure about.
From the colour scheme to game content. And you can test out two versions or as many as you want. Let’s walk through an example of creating effective AB testing scenarios. Let’s say we’ve made a platform a game where we jump around and shoot enemies to advance levels. Our analytics have shown us that only 20% of players make it past Level 3.
We think that the level might be too hard, so let’s change it. Let’s test out different levels of difficulty and see how users respond. To see how players respond to different scenarios, we can use A/B testing, a method of testing that compares the performance of two or more game versions A and B.
The most effective A/B testing scenarios are designed to test out one main variable, X, Y, count, colour, etc. And they monitor one or two results like level completion or profitability, things like that. Our game versions are distributed amongst our players. And so there are usually only 2-4 versions tested at one time.
So for our game we want to change level 3. Let’s create three different test cases where we change the number of enemies and see how many more players will complete the level. You also want to test your original version and comparison to the new ones. So we have an A, original version. A version B with 10% fewer enemies.
And a version C with 20% fewer enemies. The next step is to distribute these versions to our players so that about a third of our players play each version.
A percentage usually does this. So, we have 33%, 33%, and 34%, which adds up to 100%, distributed to versions A, B, and C. We get our results and see how many players complete the level. Only 20% of our group A players completed the class. 80% of group B completed level 3, and 70% completed level 3 for group C. Both B and C seem like a lot better options than A. And maybe the reason that C isn’t as high as B is because it’s too easy, so people get bored playing.
Either way, A/B testing lets us know for sure that version B is our best option.