A Tester’s Approach to Problem Solving

I came across an interesting logic puzzle in the New York Times today. You get 3 numbers and have to figure out the pattern:

If you like puzzles, take a moment and check out A Quick Puzzle to Test Your Problem Solving. Then continue on to see what this has to do with testing software …

When I tried the puzzle, here’s what I entered:

By the end, I was pretty sure the pattern was that the numbers increased — and I was right. But most people working on this puzzle don’t get that far, or they base their theory only on guesses that fit their assumed pattern. Here’s what the New York Times has to say about that:

Remarkably, 78 percent of people who have played this game so far have guessed the answer without first hearing a single no. A mere 9 percent heard at least three nos — even though there is no penalty or cost for being told no, save the small disappointment that every human being feels when hearing “no.”

It’s a lot more pleasant to hear “yes.” That, in a nutshell, is why so many people struggle with this problem.

In my experience, that’s also why some developers have trouble testing their software. When you write a program, you know what you want to use it for and what it’s supposed to do. And once you get it to do exactly that, you get excited and think you’re done. But users don’t always use a program the way you expect them to — start putting in unexpected input or navigating through it in the “wrong” way and you can run into trouble.

Part of what makes a good tester is a penchant for hunting for the “no.” Poking and prodding and cheering whenever something doesn’t work the way it’s supposed to. Because until you try to break it, you can’t be sure it actually works the way you think it does.