Are beer ratings biased?

It is impossible to keep bias out. I'm not a fan of pale ales in general, due the typical bitterness. Still, I make it a point to try the higher rated ones, particularly if they don't have off-the-chart IBU ratings. I like a few of them, but most I don't. Hard as I try, since I can't relate to the style, I expect I'm rating them lower due to my bias against pale ale bitterness vs rating them within the spectrum of that style.

To help overcome this bias, a few months ago I set out to try all 104 styles. I'm stuck at 97 until I can hit some out-of-state bottle shops where brand distribution is better/different. I may never get through all 104, but during this journey I have enjoyed many of the styles, and encountered only a few where I would not seek to try them again.

 

I prefer to rate to style for this reason. There are AALs out there that could be a 5/5 for the style, but when rated against things like ipa or imperial stout those numbers skew.
@bbtkd that's quite the quest you're on. It's inspiring, I may have to force myself to buy a new style each week just to break out of my comfort zone.

 

I rate based on how much I enjoy the beer. I'll usually give a bump if it blows me away for the style, but you'll still see my top beers being mostly stouts and IPAs or pale ales.

 

I don't think ratings are biased, necessarily (and depending on the system that may change) but I think that RATERS are biased, in general.
Now, Don't get me wrong, many raters will do the best they can to be fair, but there are a lot of factors that skew the ratings of many beers.
First, is the taster themselves. Do they like this style, or not? Can they be fair to a style they don't care for? Then there's the style itself. Like it or not, BBA stouts, sours and IPAs (especially NEIPAs) these days get a boost in their score for being a BBA stout, sour or IPA.
Something like a Pils may get marked artificially down, possibly unfairly, by virtue of not being a hazy hop bomb.
Lastly, I think the venue its being rated in plays a big difference. I see beers being rated at a fest all the time - and when you're drinking 2oz of something in a crowd of a few thousand closest friends, when sampling 20 or 30 beers that night, you are not going to get an accurate rating, no matter how good you think you are. Slightly better is at a bar - you may be drinking a pint, but with food and probably other beer, and with friends, again will skew your numbers.
Versus sipping a beer in your living room couch, no other distractions, you can pay better attention to what's going on there.
I also forgot to mention correcting for the condition of the beer - what was it stored, shipped, transported, how old is it, etc. Often, especially with beers from the other side of the ocean or even country, what we hold in our hands doesn't bear much resemblance to what the brewer intended.

 

Go for it, you're at 46 so almost half way there! Back in early February I realized I was at about 55 without even having tried. I created a spreadsheet listing the remaining ones, along with the better and easiest to get representatives of each. I carry that list and even ran across one Japanese style at a Chinese restaurant! I found Kvass at a Russian grocery and the owner said Kvass is not even considered a beer in Russia due to the low alcohol content. For a few weeks, all I drank were new styles. I'm amazed to get to 97 in South Dakota where the number of breweries distributed is quite limited, particularly foreign styles.

 

Rating to style is so unfair to like 95 of the styles out there. If one enjoys something, rate the hell out of it. Yet here I am with a world class lager that I am hesitant to rate above 4.25. Yet 100 other ipas get checked in at or near 4.25 without hesitation.

So yes. Biases are real. They suck and I myself find it hard to change it

 

Of course they are, the more hops the bigger the score. Delicate styles don't fare so well.

 

There's definitely a bias in individual ratings. There are those who try to limit/eliminate it, and those who don't. Hopefully, there's enough of a community here, that we balance ourselves out and get a fairly true representation (via rating/score) of a beer.

Until we reach whatever constitutes a "statistically significant" sample, beer ratings will be skewed. Best for now is just to understand it. A local IPA at 4.5 with 5 ratings probably isn't Pliny or Heady quality. An established German Pilsner at 87 is probably as good or better (within style) than an BBA stout rated 100.

Is this "right" or good? Idealistically not, but there;s probably no good "fix " or it, either.

 

The ratings on this are generally ok within the same style, but the bias for certain styles becomes really obvious when comparing one style to another style.

 

Just look at the top 100 beer list. Uh, see a pattern? I don't see any pilsners, bitters, porters, saisons, brown ales etc.

More hops, more alcohol, more haze, more adjuncts, more difficult to acquire = higher ratings.

 

This website is host to a, for the most part, specific demographic. Take a world class DIPA and a world class brown ale, which do you think would have the better overall score?

It's not really a bad thing, it's completely natural. It just means that if you're going to let reviews and scores guide your purchases on this website you should take some things with a grain of salt.

 

Yes.

/thread

 

Well...., it depends.

Biased at the level of one person or at the level of the overall scores?

Biased within a particular style or between styles?

Biased compared to what alternative?

Is a bias, if there, really worth fixing?

Etc.

The answer is: Yes or No or Maybe.

 

There actually are fixes, but most of them would "cost" more time/money/work than most people are willing take on. They would also radically change the site and it's not clear that those changes would make a more useful site.

 

Folks who are smart, like me, ignore the numbers, and look for reviews that show
-reviewer has a palate similar to mine
-reviewer gives honest reviews
and look for reviews done by people like this from whom to take my advice about beer selections. And of course smart people give reviews of this kind for others to use.

Numbers are ridiculous to use as anything except a number to complain about in a thread on BA. Reviews and number ratings have only one purpose, and that is to give information to us consumers to help us buy beer we like. Numbers, even if there were a way to remove "bias", don't do this, so it is kind of ridiculous to hear folks complain they don't do what they cannot do. Even more ridiculous to complain when there are plenty of reviews and honest reviewers whose written beer descriptions can be trusted

 

Yes... yes they are

 

Pilsner Urquell may be better then it's rating- at it's source. Not after sitting in warehouses and freighters for lord knows how long.

 

You have just defined one of the "biases". I have noticed that my first reviews of any style ( styles I have not tried I mean) seems a bit high. But I think the review is the place to really get the sense of the beer, not the number score. I think the reason you do not see a pilsner in the top 250 YET is because it's a common style. It's everywhere now and a few years ago it wasn't. The better you know the style the better you will be able to review it. I am harder on my locals and on the styles I know well. I also have noticed that most commercial beer is not that bad or below a 3 in any category. It's rare in other words to get a truly awful beer out there.