BeerAdvocate's Top 10 from 2006 is almost entirely displaced.

Discussion in 'Beer Talk' started by MichialTanner1, Feb 22, 2016.

Thread Status:
Not open for further replies.
  1. Hallu

    Hallu Initiate (0) Feb 2, 2016 France

    Maybe since 2006 the reviewers realized that St Bernardus isn't a trappist beer and that they have an industrial production ? On French and Belgian reviewing websites, the argument also seems to be that although it's the same recipe, St Bernardus uses dry yeast instead of fresh, and that they put more saccharose in.
  2. dennis3951

    dennis3951 Initiate (0) Mar 6, 2008 New Jersey

    IIRC the week after those changes took effect Westy 12 was knocked out of first place by Heady Topper.
    MichialTanner1 likes this.
  3. dennis3951

    dennis3951 Initiate (0) Mar 6, 2008 New Jersey

    There should be 2 lists. Beers that taste like beer and beers that taste like soda fountain drinks.:rolling_eyes:
    TongoRad likes this.
  4. TongoRad

    TongoRad Poo-Bah (2,988) Jun 3, 2004 New Jersey
    Society Trader

    That would only be marginally better. There really should be NO lists.
    oldbean, maximum12 and dennis3951 like this.
  5. dennis3951

    dennis3951 Initiate (0) Mar 6, 2008 New Jersey

    The key there is "according to my own tastes". If someone enjoys Storm King more the King Julius are they wrong?
    SeanBond likes this.
  6. thebeers

    thebeers Poo-Bah (4,069) Sep 10, 2014 Pennsylvania
    Society Trader

    I've had half of the '06 list, but only Westy off the '16, which is par for the course, as I'm always about a decade behind the times.
    zac16125, SeanBond and drtth like this.
  7. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    Yeah, some of those margins are so slim that it's questionable that there are any "real" differences between the scores. I mean would a score of 4.178 and a score of 4.177 really suggest that the first beer is more enjoyable than the other? (Especially when we look into how many significant digits can be obtained from the rating scales and procedures used?)
  8. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    My opinion of the current top 250 Beers List (including the top 10) is that it is a popularity contest among several enjoyable beers and such things are inherently changeable over time with changing tastes, changing candidates for evaluation, etc., etc. and can safely be ignored if we are interested in selecting from and drinking some of the best. So if you want a world class Pilsener, go the the list of Pils beers and look at the top beers their. They are much more comparable to each other than to a range of IPAs, Stouts, etc., etc. Ignore the fact that the average rating for Pils beers is lower than the average rating for, say IPAs, if only because the playing fields are different for the two styles and it hasn't been leveled for us to be able to say Pils A or IPA B is better or worse.

    If you really want the top 10 based on BAs ratings rather than the 10 most popular at the moment go to the Beers of Fame list to look for beers that have stood the test of time.
    doktorhops and TongoRad like this.
  9. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    Or alternatively the limitations of such lists and how they are formed might be better explained/understood. Some, e.g., the Beers of Fame list, the top Pils beers list, etc. can be highly useful in guiding explorations into new styles and the world of beer in general. Probably the first rule, is that such lists are a work in progress and only a road map that can be are are leaving out some of kinds of informaton which may be more important than the lists.

    Edit: Also rather than a rank ordered list I'd present them in terms of equivalence classes similar to the wording used on a beer rating page where it is said to be Good, Very Good, World Class, etc.

    I suspect that one problem with doing that with so many beers in the database is there wouldn't be a particularly obvious dividing line and so people would complain about it being arbitrary. :slight_smile:
    #169 drtth, Feb 23, 2016
    Last edited: Feb 23, 2016
    meefmoff and TongoRad like this.
  10. MostlyNorwegian

    MostlyNorwegian Initiate (0) Feb 5, 2013 Illinois

    Lists are entirely subjective, and the way the landscape has changed in those years is pretty obvious in the shifting of beers away from things that MOST people can get access to is there are a lot more horn tooters hunting for shiny objects that do not hit the wider distribution networks. Thus creating a state of being coveted. Thus creating a new desired profile in the flavor. RARITY.
    Rarity was not a desired off flavor back in 2002, like it is with the list being jacked in 2016 by shiny object horn tooters.
    It's not subjective at all for what is actually true and right.
    An allusion might be towards the average horndog who wants to bed down a hottie. Even if she or he is a terrible lay, has the personality of day old salad and is like humping a warm glass of water. I'm not saying these beers in question are like that. But. It's the shiny object status that made it happen because desire is always a bittersweet taste when you finally crack it open.
    dennis3951 likes this.
  11. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    Are you then ruling out the effects of there having been a major increase in the number of small breweries over the last ten years and at least some increased flow of talented brewers into the business that would also create lots of smaller release beers for the one-and-done crowd to covet?
  12. MostlyNorwegian

    MostlyNorwegian Initiate (0) Feb 5, 2013 Illinois

    Not at all. I think those demand that we view this need to chart and list things in a very different manner than a load dump of one top ten, and that is because of one simple reason. I can't taste that beer from a smaller brewer a thousand miles away who does not distribute, or does not produce more than 700 bbls a year. I have to physically place myself where that beer exists to have it, and I think that is the single greatest thing that this wave is bringing. The local option.
    drtth likes this.
  13. TongoRad

    TongoRad Poo-Bah (2,988) Jun 3, 2004 New Jersey
    Society Trader

    No real argument there, and I'm in total agreement with your edit. I just think it's ultimately pointless to make comparisons across styles (or even without taking context into account. )

    Unfortunately, when there are numbers there will be the temptation to crunch them; which will create the illusion that the source data means more than it really does.
    UrbanCaveman, PatrickCT and drtth like this.
  14. lhommedelamaison

    lhommedelamaison Initiate (0) Jun 27, 2015 Denmark

    A lot of interesting opinions in here, but this one I've quoted hits the nail right on the head in my opinion. The rankings make a lot more sense when you think of them in terms of total value placed on the beers by BAers; taste is a part of that, but so is the thrill of the hunt, the catch, the feast, bragging rights, trade value, monetary value, etc.

    How many people would agree with this statement: ''if the top 250 list were presented as the list of top 250 most sought-after beers, rather than the top 250 best beers, it would be substantially less controversial''?
    PatrickCT likes this.
  15. sulldaddy

    sulldaddy Poo-Bah (4,487) Apr 6, 2003 Connecticut
    Society Trader

    SUCH a flashback!! It is crazy to think about those beers and how much fun they were just 10 years ago. I think I will go find some of those this weekend and drink em again for some "beer bottle time machine" action. good thought provoking post!
  16. Angerhaus

    Angerhaus Crusader (736) Oct 1, 2015 Rhode Island

    I won't disagree that rarity leads to up-voting, but not in the way that most of you might think. Look at it this way, if you aren't into stouts then you probably aren't going to try and find a bottle of Good Morning, but big game stout hunters will track that sucker down and build a shrine to it. So we almost certainly don't have a broad base of beer drinkers reviewing whales like we do with Storm King (which is pretty much issued to every person in the USA upon turning 21).

    In that regard, I don't know that the reviews are wrong since fans of a particular style are going to get the information that they need. Maybe we should be spending more time looking at the top lists for each style instead of the overall list that doesn't represent the average, or even non-average, beer drinker.
    MichialTanner1 likes this.
  17. Kurmaraja

    Kurmaraja Poo-Bah (2,817) May 21, 2013 California
    Society Trader

    Good post, lots of interesting stuff there. I'll pick out a few things to clarify from my earlier posts and a few points of yours to slightly disagree with.

    You say: "You made two assumptions. One, that the number one ranked beer literally means that beer is the best beer in the world. Number two, that the rankings should line up perfectly with your own tastes. Both of those assumptions are absolutely incorrect."

    I actually didn't make these assumptions. But forum posts aren't a great format for deeply formal writing. I could just as easily turn this around and say "you assume I made two assumptions!" But where would that get us?

    - For point number one, "literally means that the beer is the best beer in the world". I'll say that I infer, rather than assume, that this is the loose concept of the list. The list is called "TOP 250 BEERS" and it is a global ranking. It is not called "A Somewhat Arbitrary Algorithm for Merging Beer Ratings By Non-Experts in Which 250 Happen To Rise to the Top" The implication that it is *supposed* to be a best is clear. It's also broadly referred to as such. I agree with you that it "what do the ranking mean" isn't that straight forward; that's more or less what I was pointing to. The common mental shortcut whereby many consider this ranking an objective truth is what I'm saying is silly. (Now, if you wanted, you could say "I assume many people consider this an objective truth" and, yeah, I don't have data to back me up but there's plenty of anecdotal evidence peppered through this thread ... and, I submit as additional evidence, the fact that people have paid in excess of $1500 / $2000 for bottles of KBBS + MD when reasonable priced alternatives exist ... certainly inflated expectation there that hints at an expectation of BEST.) In short, we're mostly agreeing.

    - For point two, I definitely do NOT agree that any aggregate rating with an arbitrary algo will line up with my tastes. Crowd sourced info isn't supposed to line up with that of any individual novice's or any expert's. But it is *supposed* to provide a collective "source of truth." (If not, why do it? Why debate the criteria for rating, the algorithm for combining ratings ... unless you thought you were improving the rating system). So there's a bit of inherent struggle in the point of a list like this where we're kinda supposed to assume it's not our truth or anyone's truth, but a community truth. (I'm no philosopher so there's probably better language for this in epistemology.)

    Toying with your rating game example ... one may not be able to assume that the top rated beer is THE best, shouldn't one be able to assume it is "among the very best"? Shouldn't one be able to assume that the lowest rated "may not be the worst" but almost certainly isn't "the very best"? To be concrete, at a certain "tier" you should be able to say that Morning Delight and CBS are legitimate contenders for that top spot, at least as American Imperial stouts? This is more or less the problem the tiering system - world-class, outstanding, very good, etc - tries to address. If everything is just subjective and you can't make any claims about "truth" from the list ... why the hell make it, refer to it, write long posts in forums about it? ;-) It's got to either be practically useful or theoretically useful or the discussion should just end, right?

    Following up on that, you say "its ridiculous to claim the Pliny, the CBS, the Mornin Delight, and the whatever else is up there shouldn't be there. All just because." I'm not claiming they're up there "just because". I'm claiming that there are factors besides JUST quality that helps them be up there, not least of all the algorithm that sorts the list. We saw how significantly the list changed with the algo changed. Same beers, same data, different rating. There is an element of the "just because" - i.e. the arbitrary - in it. The flip side of your statement is also true though: we can't claim that they're not there in part "just because." In fact, that's what you argued in your example of judges rating a quad, a stout, an IPA; how you choose to process the data (and which judges self-select to be at the table) are that "just because" element. Is the rating the mean? The average? Do we throw out the low and the high and average the remaining scores? Are the judges professionals? Are they rating their local favorites?

    Finally, you ask "If you really weren't taking it personal, why else would you have such a huge problem with the rankings?"

    You wrote a long post ... does that mean you are taking the ratings personally? I don't have a problem with the rankings any more than I have a problem with, say, the scoring system for international figure skating or debates about whether the 2015 Broncos or the 85 Bears have the best defense in NFL history. We're all overly interested in beer so debating minutiae becomes a way to pass the time! If anything, I have a problem with human folly - paying close to $100 an ounce for KBBS - and I'm amused / perplexed by how people take this list to be gospel.

  18. MyThoughtsExactly

    MyThoughtsExactly Initiate (0) Jun 10, 2015 Virginia

    The lists are really incomparable since the formula for calculating the weighted rating was changed. Changing the minimum number of reviews (I believe it is now ten) makes it so beers with a couple really high reviews make it on the 250 list. It makes the list more dynamic but makes it a little too easy for hyped one-off beers to make it on the list. Make a NE style IPA/DIPA or a bourbon barrel aged stout with adjuncts, get a couple high reviews, and all of a sudden your beer is world class and on the 250.
  19. bubseymour

    bubseymour Poo-Bah (3,432) Oct 30, 2010 Maryland
    Society Trader

    IMO the 1 to N rank order is a very misleading way to look at beer quality. Tier groupings are much better approach (for any fantasy football nerds you understand this logic).

    Especially once you look at like #60 to #250, how so many beers have the same rating to the 100th of a decimal.
  20. Zonk

    Zonk Initiate (0) Dec 2, 2014 New Jersey

    The change is compounded by some of these breweries making tons of new beers that are basically riffs on the same recipe. Whereas ten years ago a brewery might make an IPA, a Pale, a Stout etc, Treehouse currently has 10 DIPA's that are rated over 4.5 (in addition to 5 IPA's). I'm not sure anyone could name them all in a blind line up.
  21. David_Deas

    David_Deas Initiate (0) Jan 26, 2016 North Carolina

    No offense, but I think you're *way* overstating your case.

    Do you have any clue how many rare beers there are in the world? That never make it anywhere near the top 250? In fact, there seem to be more rare brands in the beer world than shelf turds.

    If rarity alone was such an important factor I'd expect a completely different top ten.
  22. AlcahueteJ

    AlcahueteJ Poo-Bah (1,907) Dec 4, 2004 Massachusetts

    Ideally I'd like to see something with lists per region (both in the US and the world) with users who live in those areas providing reviews. And then those lists further broken up by style.

    Beyond that, the lists could be divided by the size/distribution range of the brewery, with one list for breweries the size of Sierra Nevada, another for Victory, another for Hill Farmstead...etc.

    I would never expect this to be done, but in an ideal world this would control for style bias, freshness, and availability issues.

    Because let's be honest, it's completely unfair to compare a widely distributed, bottle-conditioned C-hop forward IPA like a fresh "juice-bomb" consumed days (or even the day of) after canning like a Weihenstephan Original that traveled across the ocean in uncontrolled environments that's already 5 months old and is a Helles and not an IPA.

    Moreover, each region has it's own style(s) that it does best. The best Koelsch is likely overseas, and the best "New England" IPA is likely in, well, New England.
    UrbanCaveman, TongoRad and SeanBond like this.
  23. dennis3951

    dennis3951 Initiate (0) Mar 6, 2008 New Jersey

    True, but they can't be so rare that no one hears about them. I have had many excellent IPA's in brewpubs that were brewed once but not added to the rotation for whatever reason. I'm sure that happens all over the country.
  24. MostlyNorwegian

    MostlyNorwegian Initiate (0) Feb 5, 2013 Illinois

    If you're going to pull a bonafides and ask me if I have a clue. I work at a brewery which only sells its beer on site, and I know plenty of other brewers whose beer is only sold in this city. Lists like these do good beer a disservice.
  25. Hallu

    Hallu Initiate (0) Feb 2, 2016 France

    Regarding the hype a beer gets and how rarity can drive ratings, there is a very good example with 3 Floyds - Dark Lord. This blind taste for this beer is great : I've watched other blind tastings from this bunch, and they're quite fair. They picked that Chimay is an average Trappiste while Rochefort 10 is clearly better (while ratings on BA and RB are not that far for the 2). For this Dark Lord it's so funny to see that their honest opinion is so far from the hype this beer gets... And if you read the reviews on BA, you can clearly see some people have honest opinions there as well while others simply push themselves to like it... They did blind tastes for Zombie Dust and Heady Topper, again recognizing that these are great beers but not 9/10 beers that are amazing and different from the pack of great American pale ales. We need more blind tastes like these with pretty normal guys. Those are not hardcore experts, those are craft beer lovers like us. I mean they couldn't pick out the crazy fruits or spices people sometimes get from beers (like mango, pineapple, pear, cardamom things like that), it's more simple (roasted, malty, coffee, chocolate, boozy etc...).
    UrbanCaveman and MichialTanner1 like this.
  26. HopBomb515

    HopBomb515 Initiate (0) Jun 15, 2013 New Jersey

    I disagree with this.
  27. vtcraft

    vtcraft Initiate (186) Apr 1, 2013 Vermont

    Sorry if this has already been said but how can anybody say Sip is better than doublesunshine. I've been apart of multiple tastings comparing both from tap and bottle/can and nobody has ever ranked sip higher than Doublesunshine.
  28. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    Side-by-side tastings often give different results than having just one beer to evaluate.
    TongoRad and David_Deas like this.
  29. David_Deas

    David_Deas Initiate (0) Jan 26, 2016 North Carolina

    Finally! Somebody understand this!

    Lining up hoppy, palate wrecking beverages in one giant session is the most unfair test.
    Fargrow and TongoRad like this.
  30. TongoRad

    TongoRad Poo-Bah (2,988) Jun 3, 2004 New Jersey
    Society Trader

    Along those lines, and since Sip of Sunshine was brought up, I have found that it does not have the complexity required to take it to the next glass (or can :wink:). Once my palate has acclimated to the initial barrage of fruits, I am left with just bitter (not so interesting) grapefruit pith as I'm starting my second one. It's not the kind of beer experience I prefer, and you are also not going to be able to tell something like that just from the first few ounces.
    champ103 likes this.
  31. JerzDevl2000

    JerzDevl2000 Poo-Bah (4,492) Oct 7, 2005 New Jersey

    Great post! I joined in late '05 and was wondering that as I looked through a list of the BA Top 250 that I printed out a few years back. It would be interesting to see how many of the Top 10 Breweries now weren't around in '06 either, if that was possible!
  32. cmiller4642

    cmiller4642 Disciple (324) Aug 17, 2013 West Virginia

    Ratings are nice for me to use as a general reference when I'm shopping (if something gets a bunch of 2's etc... I usually know it's a stinker) but I've found some great beers that people don't hype up. Fat Head's Head Hunter is easily obtainable and a world class IPA in my opinion. Part of the fun is trying new stuff rather than worrying about landing a bunch of whales.
  33. jmv0009

    jmv0009 Initiate (0) Oct 15, 2015 Alabama

    I am 22 years old, so I have a more recent time perspective and the kinds of beer that my contemporaries are drinking are very heavily weighted toward big Stouts with bold flavors (IS, Chocoalte and Milk stouts), Belgian wheat beers, Hefeweizens (very popular style with all of my friends) and IPA's. Also, I've noticed that younger BA members tend to give higher ratings and are more generous (I try to steer away from this as best I can) so this may be a reason in the current trend.
  34. zac16125

    zac16125 Poo-Bah (2,351) Jan 26, 2010 South Carolina
    Society Trader

    You don't even have to go that far back to see dramatic differences, and you can expand the sample size. The top 100 when I started on BA back in 2010 is almost completely gone (or at least completely out of the top 100). Personally, I think BA needs to adjust their ranking algorithm. IMO way too many trendy beers with low #'s of ratings are at the top. I honestly think the "Beers of Fame" list is more representative of the top beers in the world. Skimming the top 100 now, 29 of the top 100 beers have less than 100 reviews, and many have less than 50. Come on.

    I also think retired beers should be able to remain on the top beer list. Maybe I am just old and curmudgeonly and longing for the "good ole days".
    sulldaddy likes this.
  35. MichialTanner1

    MichialTanner1 Initiate (0) May 17, 2009 Texas

    Here's the link to the web archive from 2006 if you want to dig a little further.
    Jaycase likes this.
  36. MooseBoose

    MooseBoose Initiate (0) Jun 6, 2007 Wisconsin

    I like the 06 list better. I hate bourbon.
  37. WesMantooth

    WesMantooth Poo-Bah (3,161) Jan 8, 2014 Ohio
    Society Trader

    Hype/rarity certainly plays into it, but there have been a ton of exceptional new beers brewed in the last 5 years, let alone 10. I think the changes to the rating system have had a significant impact too (see FBS).

    On a personal side note, the only way Storm King would make my top 10 is if there were only 10 beers. Nothing at all against Victory, just sayin.
  38. edward_boumil

    edward_boumil Initiate (0) Jun 28, 2015 New York

    Of course they are not, but in defense of the current list its probably more than likely a lot more people like Julius than Storm King, given its higher rating. Which was my primary point.
  39. dennis3951

    dennis3951 Initiate (0) Mar 6, 2008 New Jersey

    I don't know about that, it assumes that more people enjoy DIPA's more than they Imperial Stouts
  40. papat444

    papat444 Initiate (0) Dec 28, 2006 Canada (QC)

    Ah, that's the year i joined BA. Simpler times :wink:
Thread Status:
Not open for further replies.