Tuesday, June 3, 2014

On Writing Negative Reviews


I'm a product reviewer. The primary output of my job as a writer is product reviews. Despite what you might believe, it's not at all fun to write a negative review.

The majority of products I review are software. It's actually pretty easy in that category to avoid having to write a negative review in the first place. Why? There's so much software on the market that if a product is truly and utterly bad, you've probably never heard of it and don’t need advice about whether to buy it. Sometimes poorly made software does receive some hype, in which case I will give it a negative review. But in that case, the developers are just as likely to read the bad review and release a new version of the product within days (sometimes hours).

Hardware is another story. Hardware takes time to design, prototype, announce, manufacture, sell, and ship. When a product gets that far along, it's not usually horrible, but sometimes it is, and when it is, the makers have a long road ahead of them before they can alter it. 

Poor Performers
In three years, I've rarely rated a product lower than a 2 out of 5. Off the top of my head, it's happened only twice. One of those times was just a few days ago. 

I first heard about the product months ago. I wrote about how it the idea of it fit into some upcoming trends for 2014. Then I received the product, and I talked about it and showed it off on a daily news show, mentioning that it had only just arrived and I hadn't yet tested it.

Then I started testing, and I ran into all kinds of problems. I asked the company for help and advice. I followed their advice.

I had more problems. They said the problems were occurring at the app level (in other words, a software problem and not a hardware problem, so hopefully easy to fix). It's a device that connects to a smartphone app via Bluetooth. I waited for a new app release and tested the product for several days. It worked, but it seemed like something was missing.

So I set up a phone call with the company.

We talked through my concerns. The team acknowledged that the problems I mentioned exist. They impressed upon me what the company had done right with the product, which I heard but ultimately had to ignore because none of those things changed the end user's experience.

"If there's a major app update coming in the next few weeks, let me know," I said.

I ended by telling the team that if there was no imminent update to the app or firmware, I'd have to review the product as is because it has received some hype and has been on the market for several weeks. Silence. We said goodbye.

I wrote my review and gave the product 1.5 out of 5 stars, which equates to "dismal." 

Not Fun
Maybe you've read hilarious negative reviews on Yelp or even sites with professional reviews and thought they must be a hoot to write, that it must feel great to rip into someone or something with such condescension.

Yelp reviewers and even restaurant critics don't have the same ongoing relationship with purveyors that we technology product reviewers have. Restaurant critics might dine in disguise, but I often have to meet face to face with a CEO, VP, product manager, or PR rep before the company will loan me a new product. Some of them insist on giving me a demo before putting the product in my hands. Often I push back and say that I need to experience the product with the same blank-slate mentality that a consumer would. 

On average, fewer than four companies a year still hold out, even after my push-back, and insist on meeting and demoing the product before giving it to me.

A bad review had better be factual, which means bad reviews (in my experience anyway) are more rigorously fact-checked than glowing reviews. Before I write a bad review, I have to talk to the developers or PR team. I have to make known the bad things I’m going to say by fact-checking them first. “Is it true that there is no Feature X?” I might ask. “Am I correct that there is only one way to do Y?” They know it’s coming.

And after the bad review, there's always follow-up. Sometimes the team feels burned. Sometimes they want to have a meeting with you right away to try and change your mind. Sometimes they don't approach you for several months until the product has been significantly changed. In any event, there's always a moment of awkwardness, but (with few exceptions), we're all pros and we've all been there before.

Without going into too much more detail about how we test and how many other people weigh in on the final decision about a product’s rating, even when several people support your decision to rate a product negatively, it’s still not fun to do. We reviewers know that we can’t take it back. We can update an online review when the product is updated, but sometimes we don’t have the capacity to do so quickly, and bad reviews linger. We know they can hurt a company. We know there are real people with families who run those companies. And still, when a product is worth your money, I have to say so. That’s my job. 

Hints for the Little Guys
When very small companies, the ones with products you may have never heard of before, approach me and ask me to review their product, sometimes I try to give them a few warning signs that the product is not going to score highly. This only happens in the case with relatively unknown software, the kind of stuff I would normally pass over. From time to time, a newer piece of software will look interesting and I will want to review it. But if my first impression is that it’s very bad, I might reply to the company and say, “Are you sure this product is ready for review?” or “Is the product still in beta? I don’t want to test it before it’s in its final release state.”

So if you are a small software start-up company and you hear words like that, go back to your team. Look more closely at comparative products on the market. Revise, revise, revise. Don’t allow both you and the reviewer to have to suffer through a bad review.

1 comment:

MikeS said...

Great article, Jill. Thank you. I guess asking for a review is like asking someone, "How do you like my new haircut?" We only want to hear one answer, "Looks great!".

So anyway, hypothetically, if someone were to write say a travel app for business travelers and they were interested in having it reviewed, what would be the best way to go about that?

Post a Comment