Are beer ratings biased?

Taste is subjective. I don't rate or read reviews ( gasp) The only taste that matters is my own. Drink fresh, drink local.

 

In general, no. However, when it is a beer that is not easily obtainable aka rare, then the rating becomes more susceptible to bias. It is just human nature to subjectively rate such a beer a bit higher than your normally would because the fact that you are drinking it and others are not makes it taste better. Remember, the most delicious flavor in beer - is rare and I believe this skews ratings.

 

kinda hard to not let bias creep in. that being said, I can appreciate a well structured lager as much as a stout. does one have more flavor? most would say yes. I would say that is also subjective. my favorite style of beer is properly conditioned cask pale ale/bitter which doesn't beat you over the head on the first sip. but a well made beer will keep you ordering pints and you will pull nuances out of ever sip that you didn't find previously. As long as you are enjoying it I say be as biased as you want!

 

I read a review the other day that just said something along the lines of "wow, this has a lot of alcohol. I'm still recovering." and they rated it low. How does that help? No notes on the look, aroma, flavor profile, or feel. Did it just taste like alcohol? Like boozy bourbon alcohol or a fresh, harsh wine alcohol? Fortunately, since it was a review, it was made available to report and clean out of the system. It makes me wonder how many others are dumping a low rating on a beer because they just simply don't like it. That thought makes me a little leary to lean wholly on just the numbers. I just really like how the review gives the ratings a tangible quality, especially since there are several interpretations on how to rate a beer on here despite there being site guidelines.

I do agree the numbers have a place though. It's a quick check for how a beer performs, at least within this community. If I'm on the fence between buying two different beers, I'll check their ratings first. If they're within 0.15 or so from each other, I'll read a couple reviews to see if one sounds better then the other and then make my purchase.

 

Your examples are exactly why some reviews should be ignored and, as @cavedave recommends, we should be paying most attention to reviewers who share tastes similar to our own.

 

im with you on the reviews. and to be honest...I like to look at the lowest ratings and read peoples gripes about something for the reason u mentioned. someone drinks a big 12% beer and whines about it tasting of alcohol, yeah disregard that review imo. numbers are good when there is a large sample size. im not sure id use it to pick one over another, but I will take a look (after I am already interested!) and see if its worth my money and time.

 

Everything in life is biased. Somebody's experience with the beer, like if they had it in an awesome time and place could subconsciously effect a rating

 

I rate appearance, smell, and mouthfeel mostly based on style, but I'm totally biased when it comes to taste. I like what I like. Plus I rate mostly for my own reference.

 

It definitely has a lot of bias especially if you are trading for a beer. Some like to score a beer off the top 250 and may rate it higher because it was difficult to obtain, rare, and/or the hype centered around the beer. Also, I've seen some reviews that always compare a beer to another within the same style. If using say, Heady Topper, for your "control" beer, all the other ratings/reviews will be biased.

I also agree with a lot of the responses on styles. If it's not loaded with hops, ingredients from local businesses/farmers, or aged in the best spirits, the simpler styles get a lower rating. I've seen a ton of low reviews for lagers because they do have an overpowering flavor/aroma because people don't keep style into mind.

Also, location bias. Some rate local beers higher to support their local watering hole. Experience can be another bias. If you're newer to the craft beer scene and only had a hand full of local things, you may rate/review higher because you don't know what's out there.

 

It depends what you call biased. I think most people rate on how much they like the beer, and most raters on here seem to like the taste of intense IPAs and stouts more than a pilsner or hefeweizen for examples. I do see a lot of straight 5s when someone likes a beer, and in my glass it doesn't look a 5 or feel a 5, etc to me. Everyone is different, some people will like a beer and others not so much. I like reviews that describe the actual beer and not the experience around it, and when the individual ratings seem fair. Generally I think a lot of people underrate, and I probably overrate. If a pilsner looks, smells, tastes, and feels really good to me then I will rate it as high as I feel accurate to my senses, not a 4.25 because it's only a pilsner. It's all subjective but I like to think that some ratings on here are honest and fair. It doesn't mean everyone will like or dislike a beer though, because everyone's palate and preferences are different. I usually rate overall based on drinkability and at least somewhat respect to the style.

 

And this is another way in which the ratings are, if not biased, then at least skewed. Personally, I seek out highly-rated beers of styles I enjoy - DIPA's and RIS's. Assuming the ratings are pretty much justified, I will likely rate it highly myself. Extend this phenomenon to thousands of other BA's doing the same and behold the result.

 

Yes, that's a known bias that comes with not requiring that the reviews/ratings be done blind.

But there are also a few other things at play. A reasonable number of folks will express their disappointment that the beer didn't live up to their expectations or justify the hard work or mony the spend in finding it by scoring more harshly. So this is where large numbers of reviews/ratings come into play and where the numbers can be useful. The overall ranking or mean score will be based on the reactions of lots of folks with different biases so at least some of that first bias winds up getting lost in the noise.

It's also where sorting reviews can be helpful, I'd say that the top reviewers and the lowest rated reviews are likely to provide a useful counterbalance. The top reviewers have been around the block more than once and are less likely to just heap praise on a beer when others have but where it isn't really due. The lowest scores given by reviewers/raters of the beer (and the size of the pDev on the beer's page) give a sense of what are the disappointments/disagreements, what is the level of disagreement and some of the reasons both groups offer for reviewing/rating as they did.

 

I try to rate to style, and I try everything, even beers that I don't like, so I can give my customers honest feedback, but it's tough for me with sours and wild ales, since I despise them so much

 

Please don't stop your mission with the style list on this site.

 

I'm very curious then, so a question if I may? Why do you even bother to use this, or any, site that is centered around reivews/ratings?

 

Of course ratings are biased. They always will be, even if rating "to style." It's inherent and it's fine.

This topic has been explored many times and there are multiple schools of thought on rating to style:
1) Anyone not rating to style is breaking the system and doing it wrong.
2) I'll never rate the greatest AAL higher than a great barrel aged stout because in reality there is an inherent hierarchy.
3) I rate according to taste regardless of whether or not it is to style.

None of these approaches are really wrong even if they aren't compatible. Group ratings that reflect all of the views above will be inherently more interesting than a singular approach.

Personally, I view beer through the lens of style, but I think "rating to style" is a terribly flawed idea. When @TongoRad specifically says "with style in mind" he is not being a pendant nor is he being casual with his word choice. It's a deliberate choice and it makes all the difference (for the better). You don't judge a Helles lager according to the same standard as a stout, but there is no genuine standard for a Helles lager or a stout. People tend to treat "rating to style" as a gold standard of a wide viewpoint, but it's really a narrow (or misguided) viewpoint. Rating according to personal taste is also a narrow viewpoint by definition.

 

That's good. Since the things you despise about sours and wild ale are likely the things fans of sours and wild ale love about them you are giving useful feed back to your customers.

 

Beer releases, current news, New England happenings, beer meet ups, events and beer discussions, style and otherwise. Went to Newport RI last weekend. Where to find good beer and places to eat. Here of course. Do you only look at ratings?

 

Thanks.

No, quite the contrary, I don't look only at ratings. I make a much heavier use of reviews in addition to such things as beer discussions (there have been times that the review-like comments from certain people in such discussions are all I need to give something a try). That is partly conditioned by having had, until the last couple of years, to purchase beer only on draft, only at exorbitant prices, or only by the case. So that is one reason I find reviews useful.

As an example of the usefulness I've found in reviews, few years ago I was confronted with a decision. Boulevard started distributing to PA. In my local retailer, case only at the time, I saw a very fresh case of the 12 oz bottles of Tank 7. So I had three choices. Leave it sit until after I finally found someone offering it on draft. Buy an entire case having no idea what it tasted like. Read the reviews from some reviewers whom I knew to be careful, thoughtful and have tastes similar to mine.

I came home, read some carefully chosen reviews and went back the next day to buy that case before someone else bought it (after that case it took me another six months to find a really fresh case to buy as a repeat so I was glad I didn't leave it sit very long before buying the first case). That beer is still among my favorites list and if it hadn't been for the reviews I might never have had it at all.

Similarly the place reviews are very helpful when traveling to areas where I don't know anyone, have only a day or two free, and want to find somewhere to go beer shopping or spend time having a beer. Again that has proven very useful in saving me time, money and helping me to find good beer that is fresh. (The beers are often local even when I'm not. )

 

I hope this isn't too off topic, but I'm curious given that you're from Vermont and rate based on, among other things, on appearance and mouthfeel. How do you rate the appearance of the super cloudy NE style IPAs that are so popular right now? Do you rate relative to other NE IPAs or relative to the general IPA style? Same re mouthfeel. Personally, I have had a number of NE style IPAs that I have liked (although I prefer IPAs with a little malt backbone), but I find the look off putting and would rate lower in appearance than, say, a Pliny the Elder. Taste is, of course, most important, but I am befuddled by reviews giving 5s for appearance for beers that look nothing like the traditional look of that beer style (I am presuming IPAs were initially mostly clear, but I could be wrong about that).