The Problem with Beer Scores

There's no way to make it perfect, it's still weighted heavily to the trendy and more popular styles. You just have to look at the style in relation to the scores and interpret them.

 

This seems more like an article to highlight Rate Beer's problems, rather than problems with beer scores in general, although it is quite interesting in lieu of their recent ownership change.

As per the rating system, was it borrowed from wine and spirits, or ultimately academia? Even in those other realms, ratings contain bias. There is a big push in academia now for teachers and those performing the grading to write up rubrics for each complex task so there are tangible qualities that can be attributed to why a paper or project scored as such. If we want to make a push to siphon more bias out of our scores, we should look to do the same. Then again, providing general style guidelines (as has been done) and establishing a larger and larger pool of reviewers could help even out biases from both sides (work in progress) - a benefit not available to a single teacher grading, or beer festival judge.

Side note: I am satisfied with the rating system on this site. The developers have been open about it, and yet have not divulged enough for users to abuse it.

 

Wine and whiskey drinker here as well. So you have a 100 point scale. Ever seen below say a 65? Nope. Taste is subjective, always will be. Haven't looked at ratings in years, just trust your own palate.

 

Ahhh, but to taste it you usually have to buy it. I trust BA ratings to steer me in the right direction when buying.

 

I'm old enough and have seen enough to be relatively immune to the ego massaging shelf tags that proclaim 92 points with an exclamation mark. Good Lord, you can become an instant wine snob without even bothering to read the label. I cannot count (well, I suppose I could were I not so dedicated to avoiding work) the number of times I've heard that "I only drink 90 points or better" comment from a person I know can't tell Petrus from plonk.

Beer is an incredibly diverse beverage that cannot be constrained by something so arbitrary as a 100 point system. A Pilsner is no more a Stout which isn't a Klosch which is not a Wee Heavy and so forth till we get to those journeyman players called IPAs. A celebration of life is this marvelous brew proclaiming it's brilliance with different styles to suit any occasion or time of day.

A 1-5 point system covers the territory nicely and you won't hear someone at a tasting saying,"I normally won't touch anything under 3.75 but this 3.69 is drinking well".



a

 

I have friends that fall into mainly two camps for rating on Untappd or whatever 1-5.

1- The rating is soley subjective and private to them. "my rating is for me only" (to determine future purchase perhaps)
2- (the camp I like to fall under)- How good is it in the realm of it's style. Surely there are some 5 star / cap pilsners out there.

 

I read the article as inadvertently implying RateBeer uses an A-F-inspired or 50-100 scale. The scale is in fact 0-100. The score for Budweiser, since ABI was invoked, is 0.

Despite the comparison to the wine and spirits worlds, this is completely unlike the scale of major publications. Even the worst wine could not score lower than 50 (Wine Advocate and Wine Spectator), would be simply "not recommended" if below 80 (Wine & Spirits), or wouldn't even be published if below 80 (Wine Enthusiast).

I'm sure there was no intent to mislead and love the continued transparency on BA ratings. I agree RB's system's downsides includes the disconnect on how users' 0-5 ratings get translated into 0-100 scores.

 

Whether BA's or RB's scores best inform consumers depends on familiarity with the scales. Those familiar with BA will have a sense the top 100 beers in practice range from 4.5 to 4.83 currently. It works perfectly for a frequent visitor like me. However, one does not need any familiarity with RB to instantly grasp the highest ratings are 100. If, as I agree, they have too many 95-100 ratings within popular styles, that's inflation from their formula (and arguably users) that need not be inherent to a 100-point scale

 

OK, that works for you. I trust the brewers that I know, ( a few) and what their general style/strengths are. My local stores do tastings all the time, plus flights at the brewery etc. I have so many, so good local choices that have never let me down. Just read the can? Jacks Abby, Night Shift, Exhibit A, Harpoon special releases, Cambridge Brewing, Ipswich, Newburyport, Wormtown etc. All do great stuff, no rating review needed. Just how I roll. The Mystic sour raspberry gose? Not so much, will leave that for someone else.

 

Straight, no chaser. Kudos for keeping it cold.

 

The biggest problem with BA ratings, to me, is that there aren't enough of them. I rate all the beers I drink on Untappd, because it's quick, easy, and connected to my beer friends. And the volume of ratings is a quality in and of itself. Meanwhile, there are beers in Colorado that get very few if any ratings on BA. Why? I think a lot of people have moved beyond lengthy, verbose ratings. For example I bought a beer that they released 800 bottles of, a BA Imperial Stout, and not one rating on BA several months later.

This is not a knock on BA, just an observation.
BA scores are probably the most accurate. Untappd ratings are inflated a bit -but if you subtract 0.15-0.2 points from each you get very close to BA ratings.

There are obvious and apparent flaws with RBs system, even more so now that the ownership is convoluted and controversial.

 

I thought that RB users rated stuff out of five, and that the rating out of 100 was actually a percentile thing? So a 100 is in the top 1% by average score, a 90+ is in the top 10% and so on.

Anyway, I dunno, to me there are a lot of ways that a beer can be great, so there's an inherent limitation in any system that tries to boil it all down to a single numerical score, whatever system you use. And that's fine but you have to be aware of it if you're going to cite BA or RB or Untappd scores as a source of certified objective truth.

 

The trouble with ratings is one reviewer might rate an IPA 4.5 because it's a hop bomb. Another reviewer might rate the same IPA 3.5 because it is over hopped.

 

It's a weighted average that's a bit more complicated, but you're correct that user ratings out of 5 get translated into a percentile ranking out of 100.

I agree, and often the publishers of numeric scores are among the first to acknowledge the importance of reading reviews from those with a somewhat similar palate. Even then, no two people have the same preferences. At the same time, an instantly graspable numeric score can be very helpful when one doesn't have the time or interest in examining descriptive reviews. The vast majority of beer drinkers probably fall in that category.

 

Too many cooks spoil the broth. Just follow one palate. Like mine, for instance, at SmokeBeerMinistry ... if you're into beers that taste like smoke. And use a 1 - 7 scale. My friend is in marketing research and that's what they do, because of science I guess.

 

A numerical score really doesn't tell me much about how a beer tastes. If I didn't have the time or interest to read descriptive reviews I wouldn't be here.

 

I have three problems with user numerical scores:
1) There is no 'perfect' beer (5.0)...disregard my ratings! So all "good to great" beers fall within 3.8-4.5
2) For the most part IPA's, DIPA's, Imperial Stouts rate higher than all other varieties...i.e High ABVs already have a leg up in rankings
3) Palate shift. I've had beers that taste great with the initial beer or two and then at subsequent sittings taste worse, and vice-versa

That said, I still use BA as a guide when shopping

 

Ratings and scores are meaningless to me. I try new beers all of the time, knowing most often what to expect. I also know some reviewers that have tastes similar to mine and I trust their recommendations. In all honesty, I think rating beers is over-rated.

 

I say keep doing what you do, Bros. Alstrom! I love the database and have learned how to make the numbers work for me. And the fact that BA.com is transparent about what gets done here makes it even better.