Style bias in ratings on this site

Discussion in 'Beer Talk' started by MNAle, Oct 10, 2014.

Thread Status:
Not open for further replies.
  1. MNAle

    MNAle Initiate (0) Sep 6, 2011 Minnesota

    I know this is not a new topic, but the recent threads on the GABF awards motivated this. I did a little "analysis" (OK, really just a cursory check...) of beer ratings on this site.

    Of the top 250 beers as of this morning, most are IPAs (including IIPAs), Stouts, and Porters, with a fair representation of various wild ales, lambics, and barleywines and a couple of APAs.

    There are no blonde ales, bitters, ESBs, and many, many others ... (at least not that I saw with a quick scan).

    Ambers have one or two on the list, but they are not near the top.

    If the top 250 beer list was an ecosystem, it wouldn't quite be described a monoculture, but it is characterized by species homogeneity.

    The lowest rated beer on the top 250 is rated 4.27.

    The highest rated blonde ale is 4.23.
    The highest rated bitter is 4 (and that is the Surly Bitter Brewer, which isn't really true to style anyway).
    The highest rated ESB is 4.23.
    The highest rated Kölsch is 4.21.
    The highest rated Czech Pilsener is 4.26.
    The highest rated German Pilsener is 4.11.
    The highest rated California Common is 3.88.

    I could go on.

    Clearly, there is a style bias on this site in beer ratings. What I called in another thread the "I don't like this style, so this beer sucks" syndrome.

    Either that or brewers suddenly forget how to make beer whenever the brew a blonde, bitter, or pilsener.

    I suspect this also affects this ratings within a style as well, where beers will be rated higher within a non-IPA style if they offer things the IPA drinker is looking for, as opposed to being best in style. Example: Surly Bitter Brewer being the highest rated English Bitter; Summit Unchained #16: Herkulean Woods being the highest rated California Common (although that might be due to the very low number of ratings: i.e. local bias).

    So, when looking for a beer in a non-favored style, I would not place a lot of faith in the BA ratings of those styles. And, the top 250 list should really be taken as the top 250 list of the favored highly hopped styles.
     
  2. markdrinksbeer

    markdrinksbeer Initiate (0) Nov 14, 2013 Massachusetts

  3. JackHorzempa

    JackHorzempa Grand Pooh-Bah (3,363) Dec 15, 2005 Pennsylvania
    Society Pooh-Bah

    There is absolutely no doubt that the majority of BAs favor BIG & BOLD beers.

    The bigness and boldness can be from:

    · A lot of hops

    · High ABV

    · Barrel Aging

    · A lot of sour

    · Unique ingredients (e.g., coffee)

    · A lot of funk

    · Etc.

    Needless to say but multiple qualities of the above will be appreciated as well. For example: Barrel Aged Imperial Stout with coffee.

    Cheers!
     
  4. Treyliff

    Treyliff Grand Pooh-Bah (4,983) Aug 10, 2010 West Virginia
    BA4LYFE Society Pooh-Bah Trader

    My reviews for beers that aren't stouts/IPAs/sours always tend to be higher than the average grade, because I try to review them as they were meant to be reviewed on this site, with style in mind.
     
  5. glass_house

    glass_house Maven (1,325) Jan 10, 2014 Ohio

    At what point does "preference" become "bias"? Maybe DIPA's are better than Czech Pilseners. That is obviously a purely subjective topic, and a somewhat facetious comment on my part.

    I see the topic of style bias come up a lot. My question is this--what are the theories on the genesis of this bias? What makes people "decide" that they like an imperial stout more than a kolsch? This site also clearly has an "American" bias so the ratings reflect that. But still, what makes Americans prefer any given style over another? Marketing? Popular culture? Whenever I rate any beer I try to balance my personal preferences with style expectations, but if a certain style is better to me, how could I not intrisincally tend to rate it higher?
     
  6. GreesyFizeek

    GreesyFizeek Grand Pooh-Bah (5,674) Mar 6, 2013 New York
    Pooh-Bah Trader

    I think some people only review beers that absolutely blow them away- and beers that blow people away are the extreme beers you're talking about.

    Something subtle, pleasant, and tasty, like a well made pilsner or bitter, isn't going to blow someone away, even if they like it a lot, so they're probably not going to review it.

    Personally, I review basically every beer I try, and try to rate them to style and how well they were made, not how much it "blew me away."
     
    souvenirs likes this.
  7. THANAT0PSIS

    THANAT0PSIS Pooh-Bah (2,263) Aug 3, 2010 Wisconsin
    Pooh-Bah

    From what I can tell, the majority of people do not rate to style on this site, rather, they rate based on how much the beer blows them away. It blows my mind that beers like NG Hometown Blonde or Great Lakes Dortmunder Gold/Eliot Ness are not 4.3 or above since they are both excellent and excellent examples of their respective styles. Furthermore, beers like Weihenstephaner Hefe (or everything they or Ayinger make), while highly rated, should be damn near a 4.75 considering how amazing they are within their style.

    It's just how BA is. RateBeer does things to style, but they rate everything roughly 100 unless it's offensively bad, so I can't really trust what they do.
     
  8. AlcahueteJ

    AlcahueteJ Grand Pooh-Bah (3,170) Dec 4, 2004 Massachusetts
    Society Pooh-Bah

    This is true, but if you dig a bit further, you'll notice it's not just BIG and BOLD, but BIG and BOLD flavors that are familiar and easy to discern.

    Guava, mango, peach, coffee, chocolate, pumpkin (well, more like cinnamon and nutmeg), bourbon, vanilla, sour...etc.

    I've found beers that accent Pils, Munich, and Vienna malts aren't as highly rated. And while hops do get attention, not all hops do. English ales that may highlight Fuggles, or Czech Pilsners that showcase Saaz, do not garner favorable ratings from Beeradvocates either.

    Heck, I even feel Cascade has fallen so far out of favor that something as wonderful as Sierra Nevada Pale Ale has been diminished by "ratings" over time.
     
    Roguer, JMS1512, Norica and 3 others like this.
  9. THANAT0PSIS

    THANAT0PSIS Pooh-Bah (2,263) Aug 3, 2010 Wisconsin
    Pooh-Bah

    It's intensity of flavor that makes people rate highly. A solid example is the difference in rating between Jester King Atrial Rubicite and La Vie en Rose or Montmorency vs. Balaton and Detritivore. Most of the less positive ratings for La Vie and Detrit are because raspberries aren't as intense as Atrial, but this is the entire point of the beer. It's the second runnings, so of course it's not going to be the same intensity; it's meant to be subtle. BA generally doesn't appreciate subtlety, therefore subtle styles like most lagers, bitters, blondes, etc. are not highly rated.
     
    #9 THANAT0PSIS, Oct 10, 2014
    Last edited: Oct 10, 2014
  10. steveh

    steveh Grand Pooh-Bah (4,082) Oct 8, 2003 Illinois
    Society Pooh-Bah

    For what it's worth, I have a Maibock, a Vienna, and 3 Helles in my personal top 10. :slight_smile:
     
  11. Resuin

    Resuin Pooh-Bah (2,909) Jun 18, 2012 Massachusetts
    Pooh-Bah Trader

    Yeah - people tend to rate beers based on how much they enjoy them, and I like that because then you get a "how much do I like this beer" rating, which to me, is a very useful bit of information. Of course, there are arguments to be made that each style should be rated against only itself, and that is still doable here if you look at each style list individually, and take the highest rated one as the (current) ceiling.
     
  12. glass_house

    glass_house Maven (1,325) Jan 10, 2014 Ohio

    I can appreciate that line of thinking, but then the question becomes one of the inherently "attractive" quality of a given style. For example, Hot Pockets are inferior to filet mignon. Should the "perfect" Hot Pocket be rated higher than a "good" filet just because it accomplished being what it's supposed to be? I'm obvously playing devil's advocate here, so please don't interpret my commentary as a dismissal of underappreciated styles of beer.
     
  13. THANAT0PSIS

    THANAT0PSIS Pooh-Bah (2,263) Aug 3, 2010 Wisconsin
    Pooh-Bah

    I would say in your example that the perfect Hot Pocket should be given a 5/5 in the genre/style of microwaved pizza-flavored gut bombs. A good filet would be roughly 3-3.5/5 on the scale of steaks.

    This does bring up an interesting point that I think you were alluding to, though: what about on the scale of food (here, beer)? I don't think there would be many arguments that "microwaved pizza-flavored gut bombs" are inferior to "steak," but does that mean "blonde ales" are inferior to "IIPAs"? It's a tough question, especially since I would say no, but I would of course say steak is better than a microwaved pizza-flavored gut bomb.
     
    Ngntjp and glass_house like this.
  14. KOP_Beer_OUtlet

    KOP_Beer_OUtlet Initiate (0) Jul 13, 2013 Pennsylvania

    Culturally we have a predilection for big, bold, loud, over the top anything...Monster Trucks, Football, behemoth TV's, muscle cars, and silicone implants...bigger better faster louder...we prefer "in your face" to sophisticated...this is not a judgement just an observation I make with my Ford hat held to my heart as I stand in front of my velvet painting of Elvis
     
  15. herrburgess

    herrburgess Grand Pooh-Bah (3,065) Nov 4, 2009 South Carolina
    Pooh-Bah

    This is a faulty comparison IMO. Much more apt would be to say, should a Hot Pocket (or to make it even more apt, a hand-crafted pizza) with the most intense tasting fillings/toppings be considered superior to one that uses more subtle fillings/toppings but has a better crafted dough and/or sauce.
     
  16. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania
    In Memoriam

    In context such as ratings of things preference and bias are synonymous terms.

    As for the sources of the bias, the ones that affect the top 250 list are at least two fold. E.g., Folks new to something like beer are likely to have a decided preference for strong, intense flavors. It often takes a bit of experience to come to recognize that sometimes subtle and gentle can be as good or better. Second, factors such as handling conditions and distance the beer has to travel from the brewery are typical ignored. And since the site ratings are not corrected for style differences, the "intensity" bias and/or other biases are quite visible.
     
    #16 drtth, Oct 10, 2014
    Last edited: Oct 10, 2014
    steveh and JackHorzempa like this.
  17. UrbanCaveman

    UrbanCaveman Pooh-Bah (1,866) Sep 30, 2014 Ohio
    Pooh-Bah Trader

    My rather limited observations brought me to the same conclusion. I frequently see only intense beers receiving high praise and acclaim, while phenomenal brews like Eliot or Ayinger Brau Weisse get ignored. It's as if there's a significant number of beer raters with either palates so wrecked or untrained they can't taste any flavor that isn't huge, or just more wow factor from the huge flavors.

    I'll admit I'm a sucker for a good Russian Imperial Stout or quad, but I try to rate them against other RISes or quads, and rate the more subtle styles according to what they are. Putting an Eliot Ness up against a Bourbon County Stout on the exact same scale using the exact same criteria is rather like doing away with weight divisions in the UFC and letting Rich Franklin in his prime go at the flyweights for the One And Only Championship Belt.
     
    LuskusDelph and THANAT0PSIS like this.
  18. THANAT0PSIS

    THANAT0PSIS Pooh-Bah (2,263) Aug 3, 2010 Wisconsin
    Pooh-Bah

    I don't really think they can't taste it; I think, like I said above, that they don't appreciate subtlety and balance much. I love both subtle beers like Weihenstephaner Original and New Glarus Two Women as well as insanely intense beers like Cigar City Double Barrel Hunahpu and The Alchemist Heady Topper.
     
    LuskusDelph, CTbrew32, Billet and 3 others like this.
  19. AlcahueteJ

    AlcahueteJ Grand Pooh-Bah (3,170) Dec 4, 2004 Massachusetts
    Society Pooh-Bah

    Yeah, well you're one of those weird people who likes non-imperial stout malts. I have no patience for beer "advocates" like you. :wink:
     
    Dirty25 likes this.
  20. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania
    In Memoriam

    When looking for a beer in a non-favored style, you can generally place a fair amount of faith in the BA ratings, so long as you recognize that the numbers/scale in do not have exactly the same meaning in one style category as they do for another style category (e.g., a 4.5 beer in one style category is not equal to a 4.5 beer in another category so comparing them is not a worthwhile guide). The top rated beers within a style will ordinarily be among the best available for the style (ignoring the limitations of availability etc., eg., many really good British cask ales will never get rated very much and may not even appear on the list because those who have access to them don't participate in the site).
     
    #20 drtth, Oct 10, 2014
    Last edited: Oct 10, 2014
Thread Status:
Not open for further replies.