Testing: Recipes and Web Sites
In my day job, I've been building web sites. Well, I don't actually build them, but I guide the process and oversee the development. Two out of four are done, with the other two scheduled to be in testing early next year.
Before any site goes live, there's a fair amount of testing to be done to make sure everything works right. On the most recent site that launched, we had a real problem re-skinning the blog section. It was a task that had been outsourced by the guy we had outsourced it to, meaning that from my perspective, the job was now three times removed — from me to the information systems department, to the outsourced guy, to his outsourced guy... who lives in Australia and works on a wholly different time zone than the rest of us. When the job finally got back to me, I tested it under relatively normal conditions. I tinkered with the blogging tools and pressed a few buttons I would normally press in the course of a few weeks of using the blog tools. I deleted something. I changed something. And then I hit the "rebuild" button. Everything broke. The design reverted back to the default blog style. The site was somehow calling the old CSS.
I wrote an email: "I broke the site!"
Aside from the headache of the thrice-removed, days-long fix, it was actually a rather proud moment for me. What a great bug to find early! More important, this particular bug was an indication that the developer didn't do the job properly. It took a few more go-arounds, but we finally got it working right.
Lately I've been thinking about different kinds of "testing" that I do, and it became oddly apparent how different bug-testing and software testing is from recipe testing.
With recipes, you want them to work. The real trick to testing recipes is identifying things that should be explicit but aren't. In other words, figuring out what's missing as opposed to what could possibly go wrong. For example, does the recipe indicate whether to cook something covered or uncovered? Your also looking for inconsistencies, such as an ingredients that's listed at the top in the ingredients section but never mentioned in the directions. Recipe testing is about verification.
Testing a web site and looking for bugs in software, on the other hand, is much more about trying to break stuff. The part when you're trying to figure out what's missing comes much earlier. Looking for bugs requires a different kind of focus. While recipes are more a set of instructions (and the "user" brings his or her own tools and supplies), a web site or piece of software is itself the tool. Recipes culminate in a final, measurable product, while web sites typically give the user an unquantifiable experience.
I'm sure as I think about this more, I'll start to see more similarities between the two. And maybe this essential difference that I see — how one is almost a challenge to break stuff while the other is more of a "fingers-crossed" kind of approach — is not so much a difference in the tasks themselves but more a reflection of my attitude about how I approach them.
Before any site goes live, there's a fair amount of testing to be done to make sure everything works right. On the most recent site that launched, we had a real problem re-skinning the blog section. It was a task that had been outsourced by the guy we had outsourced it to, meaning that from my perspective, the job was now three times removed — from me to the information systems department, to the outsourced guy, to his outsourced guy... who lives in Australia and works on a wholly different time zone than the rest of us. When the job finally got back to me, I tested it under relatively normal conditions. I tinkered with the blogging tools and pressed a few buttons I would normally press in the course of a few weeks of using the blog tools. I deleted something. I changed something. And then I hit the "rebuild" button. Everything broke. The design reverted back to the default blog style. The site was somehow calling the old CSS.
I wrote an email: "I broke the site!"
Aside from the headache of the thrice-removed, days-long fix, it was actually a rather proud moment for me. What a great bug to find early! More important, this particular bug was an indication that the developer didn't do the job properly. It took a few more go-arounds, but we finally got it working right.
Lately I've been thinking about different kinds of "testing" that I do, and it became oddly apparent how different bug-testing and software testing is from recipe testing.
With recipes, you want them to work. The real trick to testing recipes is identifying things that should be explicit but aren't. In other words, figuring out what's missing as opposed to what could possibly go wrong. For example, does the recipe indicate whether to cook something covered or uncovered? Your also looking for inconsistencies, such as an ingredients that's listed at the top in the ingredients section but never mentioned in the directions. Recipe testing is about verification.
Testing a web site and looking for bugs in software, on the other hand, is much more about trying to break stuff. The part when you're trying to figure out what's missing comes much earlier. Looking for bugs requires a different kind of focus. While recipes are more a set of instructions (and the "user" brings his or her own tools and supplies), a web site or piece of software is itself the tool. Recipes culminate in a final, measurable product, while web sites typically give the user an unquantifiable experience.
I'm sure as I think about this more, I'll start to see more similarities between the two. And maybe this essential difference that I see — how one is almost a challenge to break stuff while the other is more of a "fingers-crossed" kind of approach — is not so much a difference in the tasks themselves but more a reflection of my attitude about how I approach them.