Rankings (based on "mean" rating data)

Talk Discussion in 'BeerAdvocate Talk' started by bubseymour, Feb 15, 2016.

Thread Status:
Not open for further replies.
  1. jakecattleco

    jakecattleco Grand Pooh-Bah (3,749) Sep 3, 2008 California
    BA4LYFE Society Pooh-Bah Trader

    And I would say the freshness factor may add some credibility to that sentiment. Especially for beers that have to travel days for distro.
     
  2. ELS

    ELS Savant (1,053) Apr 22, 2012 Florida
    Trader

    My point is that the mean is off for a state like Wyoming due to inflated beer ratings from biased BAs living in Wyoming. I think I'm safe to assume that hardly anyone living more than 300 miles from the state of Wyoming has tried a Wyoming Beer. How many beers have you tried from Wyoming? Probably zero unless you go skiing in Jackson Hole. How many different beers have you tried from California? Probably 50 or more. Therefore, California Beers have many, many more ratings from unbiased out of state BAs that give a more accurate rating of how good the beer actually is.

    Chances are based on your example, you would be disappointed with the Wyoming Beer because on average what should be a 3.95 beer (the mean) is actually an overinflated 3.25. The California Beer on the other hand, with an average of 3.79 actually will be a 3.79.
     
    JackHorzempa likes this.
  3. SammyJaxxxx

    SammyJaxxxx Initiate (0) Feb 23, 2012 New Jersey

    Just told the wife. Forget that trip to California we were planning. We are going beer hunting in Wyoming
     
    SeanBond likes this.
  4. Hallu

    Hallu Zealot (526) Feb 2, 2016 France

    Careful, some beers with 50 ratings have fair ratings. 50 LOCAL ratings may be biased, but if you get a good sample of 50 reviewers from all over the country or the world, it's fine. A good example of that, I think, is http://www.beeradvocate.com/beer/profile/25314/199189/ Ras Soft Citra IPA - Omnipollo. 4.2 is about right, I feel it should even be 4.3. And there's only 11 reviews, 8 from the US, 1 from Romania, 1 from the UK and myself from France. I think it's a good representation of what the beer is. It's brewed in Belgium by a Swede, so 0 local has tasted it.
     
  5. bubseymour

    bubseymour Grand Pooh-Bah (4,800) Oct 30, 2010 Maryland
    Pooh-Bah Trader

    At first glance I would say there may be some logic to this. However, I just checked several of Wyoming's brewers/beers reviews and it appears that over 75-80% of the reviews for various beers (that count towards the stats), are from out-of-staters. So its safe to say that Wyoming brewers are in business thanks to tourists and not supported as much from in-staters.
     
    #45 bubseymour, Feb 16, 2016
    Last edited: Feb 16, 2016
  6. captaincoffee

    captaincoffee Pooh-Bah (2,218) Jul 10, 2011 Virginia
    Pooh-Bah Trader

    The margin of error with a sample size of 11 is huge. Feeling like the score is fair or accurate is irrelevant. That's the whole purpose of statistics.
     
  7. Hallu

    Hallu Zealot (526) Feb 2, 2016 France

    Yeah but when hesitating whether or not buying a new beer, are you gonna wait months or years before the sample size is big enough ? No, you're gonna assess if the 10 people who already reviewed it can be trusted or are overly enthusiastic with no real perspective on it. Usually when all the reviewers find the same tasting notes (ie fruits, roastiness etc...) and pretty much gave the same grade, it's ok. If you see a couple of 5's or 4.8's real early with other 3.8s, beware.
     
  8. PapaGoose03

    PapaGoose03 Grand High Pooh-Bah (6,057) May 30, 2005 Michigan
    BA4LYFE Society Pooh-Bah

    Was she disappointed? :grinning: (Nothing against Wyoming as a great place to live or visit as long as beer isn't involved.)
     
  9. bubseymour

    bubseymour Grand Pooh-Bah (4,800) Oct 30, 2010 Maryland
    Pooh-Bah Trader

    I think that's why the Bros went with 10 reviews (not ticks) before the stats should count. In their opinion that provides sufficient sample size to round things out a little, while allowing for lesser visited beers / less traveled breweries to get on the map without waiting a decade plus for their beer to hit the 10 review mark and count in statistics.
     
    PapaGoose03 likes this.
  10. zid

    zid Grand Pooh-Bah (3,132) Feb 15, 2010 New York
    BA4LYFE Society Pooh-Bah Trader

    This list is hilarious. (The data, not the work by @bubseymour )
    2 beers in the UK and Ireland that score above a 4.25 and so on.
    Well, lots of people, and that's the issue. Perhaps many BeerAdvocate site users can agree on this notion, but those are the same people whose data created the list. Would someone in Bamberg be able to argue against it? Which brings me to:
    Forgive me for being nerdy on this, but the only thing I'd argue against is the interpretation of this data. You're interpretation is using language like: "best of the best," great, decent, really bad, crappy... beer. This data is more of a measurement of BeerAdvocate user preferences than a measurement of quality or even general likability. Yeah, these can be very related, but let's not jump to conclusions of causality. So... in the example above, you'll have greater chances of grabbing a beer that many other BeerAdvocate users like. Apologies for nitpicking, but I think it's an important distinction even if most folks will roll their eyes at me.
     
    JackHorzempa and TongoRad like this.
  11. bubseymour

    bubseymour Grand Pooh-Bah (4,800) Oct 30, 2010 Maryland
    Pooh-Bah Trader

    Agree that the people doing reviews on Beer Advocate may be a little more prone to the forum brainwashing/group-think influences, but honestly I don't know if there really is any better source to this sort of beer review data than what we have here to work with.
     
  12. Tommo

    Tommo Maven (1,462) Sep 25, 2014 Massachusetts
    Trader

    I would love to see this without the weighted BA averages. JKs MVB is a 4.52, but rounds down to 4.3ish. Harsh. Same with those casey cuts 4.49=>4.14!!!
     
Thread Status:
Not open for further replies.