Beer Reviews Biased

Discussion in 'BeerAdvocate Talk' started by Jbecks64, Jun 23, 2019.

  1. Jbecks64

    Jbecks64 Initiate (43) May 8, 2016 Kentucky

    Is it just me or do people on Beer Advocate Rate beers based on how they compare to other styles instead of how they fall in the spectrum of the style they are brewed in . Mostly when it comes to lagers . Not all light lagers suck . They have to be rated in comparison to other examples not up against your favorite Dank IPA . Anybody else feel the same ?
  2. nc41

    nc41 Poo-Bah (1,833) Sep 25, 2008 North Carolina

    They are slanted, you have to account for that looking at the ratings. If you see a 90 Pils it’s a buy, 90 ipa perhaps not.
  3. bbtkd

    bbtkd Poo-Bah (2,480) Sep 20, 2015 South Dakota
    Society Trader

    People are all over the map on how they subjectively rate anything. It's from their experience, which is typically loaded with bias. There are several styles that I don't drink often, though have tried them. I'm likely to either let my bias against them hurt the score, or realize I have that bias and give therm a higher score than they deserve.

    It comes down to the fact that I really shouldn't score or review beer styles I don't like and know well. Despite that logic, I review all new beers I try, and usually if it's a style I don't like - such as IPA or sours - I gravitate to those that are highly rated, figuring that if I'm giving the style yet another chance then I should only try the best examples.

    If you want to talk about biased reviews, look at how folks review or score cult beers such as CBS, KBS, BCBS, and KBBS. While they are certainly outstanding beers, their scarcity plays off people's tendency to rate things higher when they are hard to get. Now that KBS and CBS are easier to get, I suspect their recent average ratings are down from when they were scarce.
  4. beertunes

    beertunes Poo-Bah (6,029) Sep 24, 2007 Montserrat

    You are correct. The majority of folks do not rate to style. One of the major reasons I never look at scores before trying a beer.
  5. Harrison8

    Harrison8 Poo-Bah (3,399) Dec 6, 2015 Missouri
    Society Trader

    As a one-time teacher, I look at styles as a sort of 'rubric', and I encourage others to do the same. It gives tangible reasons as to why a beer is or is not good within that style - and then the reader can then interpret if this would be a 'good' beer for them to try.

    Not to say it's an easy process by any means, but holding to a set of standards helps establish some consistency across the years of reviews/ratings vs. interpreting what is 'good' in that moment - and that's how you can help reduce bias (notice I didn't say eliminate).
    eppCOS, flaskman, brutalfarce and 4 others like this.
  6. PapaGoose03

    PapaGoose03 Poo-Bah (2,429) May 30, 2005 Michigan

    In my mind there are 5 rating styles that can be seen in the BA ratings:
    1. Rate to taste (novices/Untappd members who rate from Perfect/Very Good to 'I hate this beer's style/taste,' or somewhere in between)
    2. Rate to style (difficult to do until well past novice status and have consumed many beers in a particular style to know good vs. bad style characteristics)
    3. Rate-to-hype bias (novices who got lucky and got their mitts on a hyped beer and now must justify the effort)
    4. Rate bias as a macro hater (I assume the OP's point by mentioning 'lager'), and can't appreciate a well made pilsner simply because it carries the name of Budweiser/Miller/Coors, and unfortunately most other pilsner breweds by good craft breweries simply because it mimics the B/M/C taste/style.
    5. Homerism. Obvious definition.
    Points three and four involve bias. Point number one is a necessity by definition until a beer drinker can transition to number two. Five is going to happen to a degree by everyone, but it's minor in its occurrences in the big picture.

    So with a long explanation, bias does occur.
  7. Roguer

    Roguer Poo-Bah (4,175) Mar 25, 2013 Georgia
    Moderator Society Trader

    This is a concept that has been visited, re-visited, and positively beat to death - and really, there isn't one perfect answer.

    Is it "wrong" to rate a beer that is tastier, but perhaps a less perfect example of its style, lower (or higher) than perhaps a purer, more accurate representative of a style that is less (subjectively) tasty?

    I hate tomatoes, and love cherries. I could rate a tomato "to style" (fresh, crisp, not rotten, etc.), but I would take a mediocre cherry - hell, I would take a poor cherry! - over a perfect tomato every day. Subjectively, then, does one rank the perfect tomato higher, knowing that you would never pick it over a cherry?

    I think all ratings should be a combination of numbers 1 and 2 that @PapaGoose03 listed, above. You should be rating to style, but not throwing taste out of the window. After all, if we come up with a poop-and-fire flavored beer style that tastes like the embodiment of leprosy, you shouldn't feel forced to give a beer in that category straight 5.0s just because it's the best possible version of terrible! How you choose to balance that is, really, up to you.
  8. MacMalt

    MacMalt Poo-Bah (3,399) Jan 28, 2015 New Jersey
    Society Trader

    I agree with you 100% I've always treated beers like judging dogs at a dog show. Breeds are judged against the standard for that breed; not against another breed. Beer should be the same. I don't drink a lot of lagers. But a lager should be judged against the ideal lager - not against, say a NE IPA. I recently reviewed Industrial Arts' Summer Landscape and gave it a really high rating because it may be the best lager I've had. I just rated an Imperial Stout that I probably liked better than the lager but I gave it a lower rating because it wasn't nearly one of the best Stouts I've had.
  9. alucard6679

    alucard6679 Aspirant (217) Jul 29, 2012 Arizona

    Oh most definitely

    What really bugs me though:

    "Terrible beer, would not buy, etc etc..."
    *Rates said beer 3.0 or higher*
  10. BenHoppy

    BenHoppy Zealot (542) Nov 15, 2017 Michigan

  11. TheIPAHunter

    TheIPAHunter Poo-Bah (1,934) Aug 12, 2007 Pennsylvania
    Society Trader

    Thank you. Seriously.
  12. GetMeAnIPA

    GetMeAnIPA Zealot (584) Mar 28, 2009 California

    I concur and that’s why I look at the beer notes vs the rating. I look for specific tasting notes that I like and what I want in an ipa, lager, stout etc.

    3 magnets has a beer called one kap Kölsch. On the label there are Untappd reviews like one cap I am not a fan of lagers. Just look at the top rated beers. You don’t see subtle styles. More alcohol, more flavor the higher the rating.
    #12 GetMeAnIPA, Jun 24, 2019
    Last edited: Jun 24, 2019
  13. drtth

    drtth Poo-Bah (3,871) Nov 25, 2007 Pennsylvania

    You also need to factor in that there are some people who have higher expections that the beer deserves and show their disappointment by rating lower than they might have otherwise. Since the two sets of biases in raters tend to "cancel out" each other the average scores on frequently rated beers usually windes up being closer to the "real" score.
  14. GetMeAnIPA

    GetMeAnIPA Zealot (584) Mar 28, 2009 California

    Personally disagree that they cancel out. If you spend a bunch of money and effort to obtain said beer you want that beer to be great. I think people rate beers higher than Lower because they are disappointed. That’s the whole thing about hype and why it exists. I think more people are influenced by the perception that it should be great.
    #14 GetMeAnIPA, Jun 24, 2019
    Last edited: Jun 24, 2019
    Junior and bbtkd like this.
  15. drtth

    drtth Poo-Bah (3,871) Nov 25, 2007 Pennsylvania

    Disappointed people tend to not show it by rating higher than they might have otherwise. (There's research on how people use rating scales that illustrates that.)

    Pick a few of the beers that fall into the "whale" category and sort the ratings by lowest and quite often you'll find people who didn't give the beer high ratings and expressing disappointment if reviewing.
    bbtkd likes this.
  16. GetMeAnIPA

    GetMeAnIPA Zealot (584) Mar 28, 2009 California

    What I am saying is that I think more people are impacted by the hype and expectations and thus rate the beer higher. I think there are fewer people who rate the beer lower than expectations because they are disappointed with a whale.

    Just look at the top beers and scarcity marketing. More beers are over hyped because they are hard to get or are supposed to be the best. Thus, they don’t cancel each other out.
    PapaGoose03 and bbtkd like this.
  17. ScaryEd

    ScaryEd Poo-Bah (1,848) Feb 19, 2012 New Hampshire

    Who cares? Let people review and rate how they want. Use your own judgement when buying beer.
    chrismattlin and thesherrybomber like this.
  18. drtth

    drtth Poo-Bah (3,871) Nov 25, 2007 Pennsylvania

    Focus on one single beer. You are probably correct they don't exactly cancel out because the number of over-raters is probably larger than the number of under-raters thus the two may not entirely cancel each other out. But neither of us have access to the data which would tell us how large the effect of either subset of raters.

    But the critical point is that not all raters are subject to the overrating bias and there are those who use lower scores to show their disappointment. In addition there are those who do neither. So the average score of a hyped beer is closer to "reality" than many folks seem to realize.
    GetMeAnIPA likes this.
  19. AZBeerDude72

    AZBeerDude72 Poo-Bah (1,661) Jun 10, 2016 Arizona
    Society Trader

    Maybe Ba needs to add another item to the rating system called Style. Then folks can rate the beer from 1 to 5 based on how the beer rates for the style its in. This could aid some since you would see the scores and if the beer ranks high but the style mark is low then we know its tasty but does not meet the style, etc.
    I would honestly like to see something like this.

    MikeWard, meanmutt, shelby415 and 4 others like this.
  20. rgordon

    rgordon Savant (937) Apr 26, 2012 North Carolina

    I'll review beers and have done so for many years. Rating beers is over-rated and inherently fraught with prejudice. Describe the beer and note your impressions.
    mudbug, LeRose, cavedave and 2 others like this.
  21. mkh012

    mkh012 Defender (650) May 7, 2015 New Jersey

    It's definitely a thing. A 3.8 Pilsner is probably excellent. A 3.8 IPA? Pass. I try to rate to style, but I think the tricky thing is even a super clean, crisp, well-made Pils, for example, typically lacks a distinguishing "wow" factor.

    I think Narragansett is the best AAL, but it's not a 4.5 beer experience, if that makes sense. Overall experience is a thing, regardless of style, and that's what relegates the less "impactful" beers to lower ratings.
  22. rgordon

    rgordon Savant (937) Apr 26, 2012 North Carolina

    This is a thoughtful response. The "wow factor" is an issue that reverberates through this industry and amongst beer lovers. Being wowed on occasion is very nice, but is certainly no requirement to appreciating very good beer everywhere.
  23. officerbill

    officerbill Aspirant (266) Feb 9, 2019 New York
    Society Trader

    I try to rate on how well that beer reflects the style the brewer calls it. I admit there are styles I don't care for and I'll try to take that into account; in fact I won't rate most IPA's because of that bias. Often I'll skip rating a beer if I haven't had enough of that style to know whether or not it's a good example.

    For instance; I gave a fairly low rating to an Ommegang beer called Prost Cologne that was billed as a kölsch. It wasn't that the beer was bad in and of itself (hazy IPA guys would enjoy it), the score reflected that it wasn't a very good kölsch.

    This is a good example of where reviews have an edge over ratings. I can give a lower taste score in the ratings then explain why in the review.
    beergoot and PapaGoose03 like this.
  24. rightcoast7

    rightcoast7 Disciple (320) Apr 2, 2011 Maine

    The problem with rating to style is that an exceedingly small number of reviewers would actually be competent to do it correctly. If you want style reviews, perhaps looking at the list of winners of GABF or other beer competitions would be the best bet.

    The reviews on this or any site are best understood as a rating of “how enjoyable do most people find this beer?” With that in mind, you should always assume that ratings for IPAs, stouts, and in recent years, sours, are inflated, and everything else is deflated. Once you adjust your expectations accordingly, the ratings can still be fairly useful.
  25. Beersnake1

    Beersnake1 Poo-Bah (2,099) Aug 17, 2013 California
    Society Trader

    This sums it up. There has to be 10 thread devoted to this topic...just keeps rearing its ugly head!
    Roguer likes this.
  26. beer_beer

    beer_beer Aspirant (228) Feb 13, 2018 Finland

    I rate according to how good it looks, smells and tastes. Not more complicated than that. My general "bias" is I appreciate NA beers. Or then the bias is many don't.

    Btw I give lagers the credit they deserve (if they deserve :wink:)
  27. tzieser

    tzieser Meyvn (1,040) Nov 21, 2006 New Jersey

    There is this one longtime, Poo-Bah-level user here with thousands of reviews to his name that I will not name outright. Anyway, he fully acknowledges his distaste for most lagers, sours & IPA's but for some reason he reviews pretty much all of them anyway, giving them scathing reviews for being bitter and 'perfumy'. He seems to love pastry stouts though and gives them stellar scores just about every time.

    Go ahead, review however you want. Obviously that is your prerogative. What I'll never understand though, is knowing that you don't like something and yet you continue to go out of your way to drink things you do not enjoy. For example, I do not like Sahti and Gruit-style beers but won't see me giving 20 word reviews for a beer that I know I am gong to give a 2/5. It literally adds nothing of value to the site or the review score (if anything it is detrimental to the site/review score IMO).

    "Oh cool, the dude that hates hoppy beers and lambics just gave Carton Boat a 2/5 and Hanssens a 1/5. I am shocked. SHOCKED I tell you."
  28. Raime

    Raime Meyvn (1,292) Jun 4, 2012 Korea (North)

    My favorites when I'm scrolling are those guys who signed up to make a one off.

    " Something something neckbeards man this hoppy beer is nasty! something something Bud Ice 5/5 Drink real beer! "

    By John Q. Public last online 3 years and 9 months ago with his one and only review.
    beertunes, maximum12 and neenerzig like this.
  29. Gassygunslinger

    Gassygunslinger Initiate (35) Dec 20, 2012 Maine

    Devil's advocate:
    What if the problem is trying to put something that is multifaceted onto a 1-dimensional number scale? There are so many dimensions to a beer, so many possibilities, that putting them on a 1-5 scale on whether they liked it tells you little.

    Think about it. Ignore the numbers, but focus on the flavors described. Horseblanket, tropical flavors, funk. Are these positive or negative terms? Neither, but they're a good directory for making decisions on whether or not you will like this beer. Tart, slightly salty, light-bodied. Do these things appeal to you? If not, stay away. Sweet, heavily-bodied, clove-like esters. Are you in a mood for those things?

    Now here is my big unwieldy impractical idea... Have a two layer review system... In one, you choose flavors, not dissimilar to how does their reviews (there are little icons for peat, toffee, vanilla, etc., and each can be scaled). Then, attach a number review on your personal preference for the beer. Then, weight the preference reviews based on people who rated high beers similar to the beers you've given good reviews. Sort of like a Netflix of beer. If you're a hop-head and love the dankest IPAs, you'll be able to see what the top suggestions are from your fellow hopheads, but won't be getting suggestions from the latest Allagash sour if you're not into those things.

    I know, completely impractical to do in a preexisting site, but a man can dream, right?
    stevepat and mkh012 like this.
  30. CTHomer

    CTHomer Poo-Bah (2,644) May 23, 2014 Connecticut
    Society Trader

    It's not too difficult to pull up the BJCP guidelines when you're writing a review of a style you're unfamiliar with. A novice can drink 20 pilsners and still not know what to expect from a good one. With the guidelines in hand, a novice could have the information necessary to write a reasonably informed review right away.
    Roguer and beertunes like this.
  31. DVMin98

    DVMin98 Poo-Bah (2,936) Nov 1, 2010 North Carolina
    Moderator Society Trader

    People base one type of beer vs another. Basically how they like it vs all other beers. I tend to base my on style.

    ie Olde Mecklenberg Copper. I rated it a 5 across the board because its the best Altbier I've ever had. Also OMB Munzler's I rated a 5 across the board because its the best Vienna lager I've ever had.

    That's how it SHOULD be done, but everyone has their own way of doing things.
    brutalfarce and officerbill like this.
  32. CTHomer

    CTHomer Poo-Bah (2,644) May 23, 2014 Connecticut
    Society Trader

    What would you do if you tasted a better alt?
  33. DVMin98

    DVMin98 Poo-Bah (2,936) Nov 1, 2010 North Carolina
    Moderator Society Trader

    I have the ability to edit my ratings if I think its necessary. Or I could have more than one five if I think they are similar. I have a few 5's in stouts, etc
    beertunes likes this.
  34. beersampler6

    beersampler6 Meyvn (1,084) Apr 4, 2018 Michigan
    Society Trader

    I too see this all the time. I don’t get why they even bother wasting time to sign up.

    To be a shill for that particular brewery?

    Originally intended to rate and review more beers, but forgot their login and password?

    Just buzzed one night after a few hours of drinking and wanted to troll online?
    Raime likes this.
  35. rudzud

    rudzud Poo-Bah (6,164) Apr 28, 2010 Massachusetts
    Society Trader

    I see a fair amount of trolling/abuse from out of staters. Give their home brewery 5's across the board but practically 1's to neighbor state breweries. Why? I mean, you're not even being sly about it as what you're doing is blatantly obvious.
    beertunes likes this.
  36. chipawayboy

    chipawayboy Devotee (422) Oct 26, 2007 Massachusetts

    Reviews biased. No way.
    bmugan, beertunes and thesherrybomber like this.
  37. Longhorn08

    Longhorn08 Aspirant (225) Feb 4, 2014 Texas

    I rate on if it looks, smells, tastes, and feels good to ME. That’s what a personal rating should be.
    BrewmanCapote and JimboBrews54 like this.
  38. Longhorn08

    Longhorn08 Aspirant (225) Feb 4, 2014 Texas

    Disagree. Best representation of one class does not always deserve the same grade as the best representation of another class. When you drink the best malt liquor does it deserve a 5 like the best Quad? Me thinks not.
    Raime, bmugan, maximum12 and 2 others like this.
  39. Sam671217

    Sam671217 Aspirant (239) Feb 8, 2017 South Carolina
    Society Trader

    I try to look at Beeradvocate ratings rather than untappd because in order to review on here you have to go on the site and there's a little more effort you have to put in. Untappd is basically social media at this point and contains way more bias in my opinion.
    The other thing with Untappd (and correct me if I'm wrong) is you can't go back and edit a scoring of a beer. You have to score it again separately. On here I can try the beer again and simply edit my score. This is an issue because everyone starting out gives stuff wild ratings. I mean KBS is one of the first beers I scored on here and I had to go back and edit it after I realized it was ahead of Handjee and Chemtrailmix on my list and it's easily not better than those.
  40. DEdesings57

    DEdesings57 Devotee (498) Aug 26, 2012 New Jersey

    In my opinion a beer should be reviewed based on the style that it's trying to represent and the reviewer should be familiar enough with the style to look for specific details regarding it.

    The reviewer by no means should never interject any biased opinions and should once again fall back on style-guild-lines for the beer in question and of course the brewers notes for any deviations.

    All beer "Should" be reviewed this way.
    Bitterbill, CTHomer and PapaGoose03 like this.