The Problem with Beer Scores

Yep. Also, there are those that are "hard graders", that are loathe to give anything a high score, whose "great beer" barely gets a "4", if that, and those that think lots of beers deserve the magical 5. Have to take that into account.

For me the # is a starting point. Occasionally, I'll look at ones I know, then look at "Beers" of those that have a similar opinion, then look at those rated/reviewed beers of theirs that I've also had. From that, I've noted those who frequently have the same general opinion as I do. Now, when I look at an unknown, I'll look for their rating/review. The daily WBAYDN? thread has been helpful in this regard.

That said, I'll try anything once, provided I can get a single.

 

I'd say this site does just about as well as you can with the data that is available to them. Some things to ponder would be that if every beer drinker in the US joined BA and rated their favorite beer the overwhelming number one winner would be Bud Light.
Another thing I wonder about is why there is no "top beers" list that takes say the top two beers in every style and calls them the best of the best.

 

Another thing that needs addressing IMO is that if you are going to carve out a new "Style" like NEIPA then you should also create a listing for best NEIPAs don't you think? https://www.beeradvocate.com/lists/style/116/

 

The BA system works for me and I can look through the ratings to get a feel of where they are coming from . Its not perfect but I understand it. Rate Beer is harder to follow and I distrust the AB InBev involvement. Any rating system has to assume a certain level of intelligence on the part of the person making decisions on the rating system. Intelligent users will understand and know what the ratings mean and how to use them in a meaningful way. For the unintelligent - we can do much about that other than hope they eventually learn and be kind in the interim.

 

The problem isn't so much the BA scoring system itself as it is the assumptions about made about what the numbers can/do mean. (E.g. lots of folks automatically assume that the mean must be 3 because it is in the middle of the numbers used in rating and/or they assume that the numbers have the same meaning across styles as they do within a particular style.)

 

My sentiments exactly.

 

Whenever I look up a beer the scores don't mean nearly as much to me as the written reviews. And while I really do appreciate the we written ones that make me feel like I'm experiencing the beer, I find it most helpful to look for common threads throughout the page of reviews.

 

The only thing my very similar use would bring to this description is that the numerical scores and lists can be very helpful as a rough initial filter. Within a single style list the 5th beer and the 10th beer may be effectively indistinguishable from each other except possibly in a bit of flavor profile. But I don't think much is lost by ignoring the bottom of the list (which often has a weakness in the flavor profile) when there are quite a few above them remaining untried. Similarly there are so many beers I've not tried it's hardly worth worrying about beers not in that style's list unless there's a special case involved.

 

The only issue I take with beer ratings is the subjectivity regarding mood, time of day, other things going on around the time of the rating. Did the rater just smoke a cigarette for instance, or are they drinking the beer with food? Someone could be having a really good day, drink an above-average beer, and consider it world class just due to thought association.

I find myself often going back, also, after years of ratings and retrying a beer. I am then forced to edit my rating as my tastes/pallet have evolved to detect and describe the new things I see in it. I still love rating - something about scribbling notes and cataloging my beer adventures as I get older is super rad.

Love going back and seeing what I was discovering a year, or two ago. Can only imagine the nostalgia five, ten years down the line. Cheers guys.

 

I would say the majority of people that rate and review beers here on BA don't rate them with the particular style in mind. One of the main examples that come to my mind are AAL's. I'm not the biggest AAL fan but there are some examples in the style that are much better than similar beers, yet they are still scored poorly because they are what they are.

 

Look at the Lager the Pils scores, even the popular Fest style beers suffer here. It takes the best of the best to get over 90 in a lot of cases, and a middling IPA probably hits that mark.

 

I would wager a bet and say that most malt-forward styles aren't rated as well as their hoppy counterparts.

 

I'd say more the wine and spirits based on how far back the BA basic rating scale ideas go in time. But I'd also suggest that both incorporate some "inappropriate" assumptions about how numbers work and what they mean. So effectively there's a good bit of pseudo-science involved in each, since neither of them appear to be solidly based in, or derived from, the scientific aspects of rating scale development or psychometrics.

 

A lot of people in this thread have said something to the effect that ratings are meaningless or that they don't pay attention to them. I don't agree with that view, but I do agree that a lot of the flaws discussed are real.
If you are a critical thinker and can factor in(or out) the flaws with any such system, then there is value in ratings. Just look at the ratings with your personal grading curve and never take then at face value. They are imperfect, but they are data and therefore of some value when analysed critically. The new inclusion of histograms on the BA ratings are a big help in interpreting ratings.

 

Seems like only ipas stouts and imperial styles get high grades

 

Given that there are top 100 lists already generated for every style here, has the site ever considered including this information on the front page for each individual beer when applicable?

For instance, the #1 German pilsner (Hill Farmstead Mary) has a rating of 4.1 rating that only earns it a #2313 ranking overall. But if the page for Mary included the information about it being ranked #1 within its style that might mitigate the issues with grade inflation/deflation for certain styles that always get referenced in threads like these.

 

Granted that any individual evaluation of a beer is subjective, and can be affected by many outside variables (time of day, mood, that handful of M&M's you just scarfed down). And granted that the quality of reviewers is highly variable, ranging from the true connoisseur to the guy who's had a few too many and just decides WTF let's write a REVIEW here. And some reviewers review to style, while others rate on their own personal enjoyment - so we're not even rating things on the same criteria. BUT...

Ever hear of the "wisdom of the crowd"? It's been proven that the average of the guesses of a large number of people (say, in guessing the number of jelly beans in a jar) is always more accurate than any individual guess. Four conditions apply. There must be: (a) true diversity of opinions; (b) independence of opinion; (c) decentralisation of experience; (d) suitable mechanisms of aggregation. Any of the rating systems (whether it's 0-5, 50-100, or A-F) meets those criteria, as long nobody (eg, RateBeer) mucks with the numbers.

So I'd contend that the larger the number of reviewers, the closer the aggregate rating is to the "true" answer. And I'd even go so far as to say that an excellent Pilsner's score relative to a mediocre IPA accurately reflects the current state of affairs with respect to those styles.

 

It reflects the attitudes of the scorers, not the beers.

All the number crunching in the world doesn't fix GIGO.