Yes, I know, I know. I am obsessed with testing. I wrote a ton of articles about this already. I spend huge part of my seminars about design talking about play testing. This is important part of design process.
Here I am today, once again, talking about my recent play test. This time - surprise! - I was play testing First Martians.
It was Thursday. Rindert and Corina were able to play test First Martians for the first time. I explained the rules of the game and of the particular scenario, sat them with Szymek, one of my trusted play testers and I left the office. I had some things to do.[Yes, me, the guy who states he watches and attends every single test of his games. I just left the office.]
I came after two hours. Szymek was white as a sheet. 'Ignacy, I screwed up. We forgot about Low battery rules. This is special rule of this scenario, I totally forgot. They won, but it was easier without this rule. I am very sorry. I screwed up.'
'You were not testing. You were learning the game. You will play test tomorrow' I said with a smile. I didn't care about this game. It was not a test. It was not play testing. It was just playing the game. I didn't care about small details like making game easier or harder and such stuff. I was preparing for real test.
The real test took place the day after. I set up the game and put new scenario on the table. 'Today you'll play different scenario.' I said. 'The question I will ask you after the game is over is simple: Is it different from the one you played yesterday?'
And they played. My employee was with them the whole time carefully observing the whole game and players reactions.
Next day I asked them. 'Was it different?'
'It was very different. It felt really different.' Rindert said. Corina confirmed.
'After playing first scenario and second scenario are you eager to see other scenarios and see what's more in the box?' I asked.
'Yes' Corina answered without hesitation.
Generally I don't trust play testers. They tend to lie a lot. I called my employee and asked for the report.
He confirmed what they said. They acted different, they focused on different aspects of the game, they felt different emotions. These two scenarios are different enough.
Goal of the play test was met.
Answer for my question was given.
I could prepare another test...
I hear so many times young designers saying things like: 'We play tested the game 200 times.', 'We had 300 play test games', 'We were play testing for 3 years.'
It's not about numbers. It's not how many times you played the prototype. It's all about questions you asked. It's all about goals you set for the particular game. It's about objectives you met in this particular play test.
Each time designer sits to play test the game he needs to set a goal for this test.
Otherwise he is not play testing. otherwise he is just playing.
It's quite a difference.