Post Format

A Tester’s Approach to Problem Solving

I came across an interesting logic puzzle in the New York Times today. You get 3 numbers and have to figure out the pattern:

New York Times Logic Puzzle

If you like puzzles, take a moment and check out A Quick Puzzle to Test Your Problem Solving. Then continue on to see what this has to do with testing software …

When I tried the puzzle, here’s what I entered:

New York Times Logic Puzzle Guesses

By the end, I was pretty sure the pattern was that the numbers increased — and I was right. But most people working on this puzzle don’t get that far, or they base their theory only on guesses that fit their assumed pattern. Here’s what the New York Times has to say about that:

Remarkably, 78 percent of people who have played this game so far have guessed the answer without first hearing a single no. A mere 9 percent heard at least three nos — even though there is no penalty or cost for being told no, save the small disappointment that every human being feels when hearing “no.”

It’s a lot more pleasant to hear “yes.” That, in a nutshell, is why so many people struggle with this problem.

In my experience, that’s also why some developers have trouble testing their software. When you write a program, you know what you want to use it for and what it’s supposed to do. And once you get it to do exactly that, you get excited and think you’re done. But users don’t always use a program the way you expect them to — start putting in unexpected input or navigating through it in the “wrong” way and you can run into trouble.

Part of what makes a good tester is a penchant for hunting for the “no.” Poking and prodding and cheering whenever something doesn’t work the way it’s supposed to. Because until you try to break it, you can’t be sure it actually works the way you think it does.


Posted by

Excellence Wrangler for Automattic, the company behind Linguaphile and Translator. Tester.


  1. There’s a great mention of this puzzle (not this NYT article, but this puzzle) in The Black Swan by Nassim Nicholas Taleb.

    Liked by 1 person

    • I’ve heard you mention The Black Swan enough that I’m convinced — I’ve added it to my reading list. 🙂

      Liked by 1 person

  2. So true! That’s what I love about getting out new products. Forget the manual, what happens when I press that button? What is that? Can I make it do that?

    That’s how you really get to know a new product! Trial and error!

    Liked by 1 person

Comments are closed.