The Verge And Smartwatch Scoring

Today, Executive Editor at The Verge, Deiter Bohn, posted his in-depth review of the LG Watch Urbane, LG’s second round-screened smartwatch, which sells $349 dollars. I read the review, which highlighted the various pros and cons of the product as reviews tend to do, and then I noticed something interesting at the end of the article: A 7.3 overall score.

I wondered how that compared to their Apple Watch review, so I did a quick Google search (don’t bother using The Verge’s built-in search function — it’s horrible) and noticed that editor-in-chief Nilay Patel gave Apple Watch an overall score of an even 7.

Skimming through both reviews, I couldn’t help but think that if I gave each of these reviewers the choice of picking the LG Watch Urbane or the Apple Watch, both would obviously pick the Apple Watch. In fact, one of Bohn’s conclusions was that the Urbane just wasn’t for him. He decided to buy an Apple Watch.

When I asked Patel about what seemed to be odd scoring, his response was this:

It’s fair to say that a newer version of Android Wear might be better than the current version of Watch OS, but the idea that different reviewers means that the numbers should have different meanings is patently ridiculous. It undermines the entire system of numerical assignment in the first place. If one smartwatch is rated a 7 and another is rated an 8, you’re essentially declaring that the 8 is better. Expecting me to weigh each number based on who the reviewer happens to be just isn’t all that ideal. Lots of people don’t even read bylines. If the reviews are subjectively arbitrary, non-arbitrary numbers probably aren’t the best way to demonstrate a comparative conclusion.

All this makes me think we should get rid of “final scores” altogether, at least over here at WatchAware (which should be easy since we haven’t used any yet). A better system is a simple list of benefits and detriments weighed against explained “common” scenarios, with a nice big caveat emptor as the only real objective advice.

Also, I’ve played with the LG G Watch R (and its old Android Wear OS), which The Verge scored identically to Apple Watch. Sure, smartwatch expectations have likely changed since the G Watch R came out (due, no doubt, to the much larger expectations everyone had and has for Apple Watch), but if the Apple Watch is a 7, than the G Watch R is more like a 5. And that’s a generous 5.

All that said, I’m not sure Patel is being totally honest about The Verge’s grading system and its reviewer-by-reviewer subjectivity. I think it’s a lot more likely that The Verge — and everyone else — is holding Apple to a significantly higher standard than the rest of the market’s players. Where Android might be docked half a point for some glaring omission, Apple loses twice that for the same exact one.

In either case, The Verge’s review system is buggy. I give it a 3.