Playtesting
|
Welcome back! This week I will be talking about the two playtesting sessions that we had during the Shoot’em up project. Since the two tests were several weeks apart, the post will consist of two parts – one for each test. I will also talk about how we incorporated the data we received from these tests. Finally, I will also discuss why and how we chose the questions we asked. Playtesting session #1For the first playtesting session our game was in a very fundamental state. We had no finished art assets, a few sounds and broken hitboxes. There was no level design or time for balance. In other words we were well aware of the fact that the game was unfun and broken at the time. I took it upon myself to create the form that we would present to the testers after each test. I did some research and looked at the links for helpful tips to consider when having a playtesting session (provided by the school). My conclusion was that many of the questions they asked in these examples would not give us helpful data at such an early stage in our project. I therefore decided to create something that was tailored for our situation. For example asking “When did you feel the most clever?” did not make any sense, while a question such as “Do you think the kamikaze enemy moves too fast/slow” would actually give us data that could be incorporated and changed based on the feedback we received.
The 7 questions we asked in the first playtesting session The questions are all very specific and we did not offer the testers a chance to give more detailed written feedback. This was a conscious decision, since the game was in such a broken state. We were all well aware of what needed to be added to improve the game and the experience for the players. It was just too early to ask such questions. Instead, by asking the questions we asked we got some really helpful data. The code worked more or less fine, but as previously stated the art was all placeholders. So even at such an early state that our game was in, we could implement the data and we still use the same numbers to this date. Going through all the data and providing pictures for all of them would result in a text that does not fit the blog format. I will therefore talk about the ones I feel were the most interesting. For the question “Which control scheme did you prefer?” 95% preferred W S + Arrow key up and down. I personally was a fan of having A and D for steering the cannon. To me that helped me differentiate betweem the shield and the cannon easier, since the two inputs are opposite in a way (left and right, instead of having both be up and down on the keyboard). However the result was so conclusive that I accepted that my preference was not the norm, and therefore not the right control scheme. 65% thought the kamikaze enemy moved too fast. We decreased the speed and as a result the game was suddenly in a much more enjoyable state. We also had 3 different crosshair options in this particular build (laser, “regular” crosshair and no crosshair”). The result ended with a 50/50 between the laser and no crosshair, in terms of which one the testers preferred. We therefore removed the regular crosshair and started working on an “easy mode” which would incorporate the laser sight to assist with aiming. All in all the first session was a massive success. We had 20 people testing our game and almost all the data we received turned out to be useful in some way. It has since been incorporated into the game and has made the game better as a result. Naturally, certain features were deemed good enough by the testers and remained unchanged, so I chose not to discuss those questions in detail. Playtesting session #2For the second session our game had improved massively. I finally felt comfortable with asking more detailed questions and have the testers provide written feedback. I had also learned from the previous playtesting session that too long forms had an adverse effect. You naturally want a lot of details and data from the testers, but by making the form too long you will receive a much smaller sample size. There was even one particular group that had 2 forms, and they were mostly asking for written feedback. Just filling in all the details took a significant amount of time, and I even had to wait for my turn before I had the chance to share my thoughts. This created an obvious bottleneck which I wanted to avoid. I therefore tried to find a healthy balance between ABC questions and questions that asked for written feedback.
The questions we asked for playtesting session #2 I was very pleased with how the form turned out. Especially the first question provided excellent data. At the time we knew we only had time to add 1 more feature, no more. It was therefore interesting to pick the brains of the testers and see what they would have added. We are all game designers, so I have great respect for the opinions of the testers. We therefore took the data we received very seriously. One tester answered (if he could add 1 more feature)” “More ways to deal with the smaller enemies, I feel quite weak” We solved this by having the small kamikaze enemies fly in a triangle formation. Not only did we not have to add another weapon or special attack, but we also found a mechanic that made sense in relation to our aesthetic goal, which was playing should feel like operating heavy machinery. It made tons of sense to have the player fire an attack that would destroy not just 1 enemy, but formations of several enemies. For every good suggestion we got, we had one suggestion that went heavily against the concept document and the aesthetic goal. Still, we were more than pleased with the result in the end. For further playtesting sessions (in future projects) I will make sure that the testers really understand the aesthetic goal and that certain clunky mechanics we had were not there by accident. These mechanics conveyed the aesthetic goal excellently and 94.4% of the testers answered yes on “Do you think that the aesthetic goal of “feeling like you are operating heavy machinery” is present in the game?” So while some of the testers wanted certain mechanics changed, they still agreed with our decisions of having them, even though they were not fully aware of it. “Is the game too hard?” Was one of the more interesting questions. We observed a massive skill discrepancy between the different testers. Roughly 60% thought that the game was too hard, while another 35% were fine with the difficulty. Certain individuals had enormous problems with the control system (basically keeping track of 3 different things), while others learned very fast. Since we did not want to change the mechanics for firing or controlling the cannon, we decided to create two difficulty modes. An easy mode which gives the player more health and a laser sight to assist with aiming, and a normal mode which is basically the version that the testers were playing. Finally we had an option for the testers to provide other feedback. By keeping the form short and efficient, we got a lot of detailed responses and also a sample size of 18 testers. I will briefly go through some of the feedback. I feel like the game should start at a lower pace and should be easier from the start, slowly adding challenge and difficulty. We were already aware of the fact that the game just started with throwing everything at the player. For our beta build we finally had the pacing fixed, and it was nice to see that the testers felt that what we were working on was high priority as well. Clearer UI We have since created large bars for the energy, health and power-up charge. For the build the testers had, these were small placeholders that were hard to even spot on the screen (they blended into the background) It was hard, maybe add different levels, easy, medium, hard As previously stated, we decided to go this route. Still, it was nice to see that the testers had the same idea for a solution in mind. The spawners are really annoying, hard to hit and I don’t feel like im progressing. I would like more enemy animations to make it more alive. The UI should be on the left side of the screen and a lot bigger. Something visual when you get power for your power-up. In our current build we have decreased the speed of the spawners. We have implemented animations for everything (except the power-up). The UI we decided to put in the top right, but we agreed that it should be a lot bigger. We want to have some kind of visual indicator for when the power-up is ready, but we are down to just 1 graphics student, so it is a matter of priority and time at this point. I like the machine, it looks very nice and detailed, but I didn,t really understand what I was doing and why. We now have a finished narrative and an intro sequence that explains the controls and the story to the player. We agreed that just having the player get thrown into the game with no explanation made no sense. Spawners move too fast, there are too many kamikazes to handle and I end up feeling overwhelmed. Also the lack of feedback gives no sense of progression and completion. This has been talked about before in the blog, but this is a great example of excellent feedback. I felt like I could “Delay” the game forever by focusing the small enemies. The small enemies didnt feel like enemies, but projectiles. The mech tank looked amazing. To progress in the game you have to kill the big “spawner” enemies. We knew that we could not change this dynamic, this is just something we have to accept. As a solution we will be trying to add a scoring system to encourage more skilled players to delay the game if they want a high score. Delaying the game would be a risky decision and only something a high level player would consider doing. It is a very minor, subtle change but it has a positive effect for certain players (while at the same time not harming other groups). All in all, the playtesting sessions turned out great for us. The data we received was very useful for helping us prioritize certain features and details in the game. I want to thank all the people that helped with testing our game and that took the time to provide useful, detailed feedback to us. I also want to thank you for reading!
 |
