Welcome to BeerAdvocate!

Looking to talk, rate or trade beer? Join today by creating your free account.

Dismiss Notice
Curious about recent updates to the site and app? Check the Announcements forum for updates.
Dismiss Notice
Love Belgian Beer?

Join us Sep 17 in Portland, Maine for Return of the Belgian Beer Fest, featuring hundreds of authentic Belgian beers and Belgian-inspired offerings.

Tickets + more: beeradvocate.com/belgian

Updates: Top 250 Beers & More!

Discussion in 'Announcements' started by Todd, Jan 19, 2013.

  1. shuajw

    Joined:
    Aug 12, 2007
    Posts:
    320
    Likes Received:
    1,238
    Location:
    Georgia
    Thanks, I know there's probably many retired beers that aren't really relevant anymore, but some like Duck Duck Gooze, M, BCBS Rare, King Henry, etc that still get talked about quite a bit, it would be nice to see them on a list.
     
  2. Beerandraiderfan

    Joined:
    Apr 14, 2009
    Posts:
    1,359
    Likes Received:
    1,385
    Location:
    Nevada
    Yeah. But it seems you want rarity to count just as much, if not more than taste?

    At the same time, I'll acknowledge my preference structure in this example is the same as yours, Kaggen > Abyss (especially the last two years of it and their whopping "6% bourbon barrel aged" Abyss they're charging 10%+ more for lately, sheesh).

    But I think if you make a 5 gallon batch of awesome beer, but only make it every leap year, and it is more elusive than Manti Teo's girlfriend, you should probably take a slight hit in the "greatest beers of the world rankings."

    I felt the top 100 list with only a 10 review minimum was a de facto "rare beer" list.

    Still not keen on "hads" counting as much as those of us who contribute a substantive review, and the lack of other information that has been missing for about a year now, as drabmuh points out. Patience is a virtue.
     
    cavedave and Rutager like this.
  3. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    The only real way this list would work is if we put a freeze on reviewing all beers that are retired; not just beers from closes breweries. Otherwise the list would continue to change based on reviews from memory or even worse ... reviews of vintage/old beer. Regardless of the list, it's something we're highly considering.
     
    yourefragile likes this.
  4. sukwonee

    Beer Trader

    Joined:
    Dec 13, 2011
    Posts:
    1,076
    Likes Received:
    1,305
    Location:
    Washington
    While you do have a point that being wide distribution shouldn't equal higher rating, a list like this needs to factor in some sort of weight-averaging to at least eliminate the potential regional biases (for example, I know 100 people in Seattle that think Mac Jack's African Amber is the greatest beer in the world).

    It sucks that Kaggen and RareR Dos took a big hit from this new rating system, but I, along with the majority of members on this site, would probably never find out whether Kaggen tastes better than Abyss and think Abyss is one of the best tasting stouts.
     
  5. yourefragile

    Beer Trader

    Joined:
    Dec 16, 2006
    Posts:
    114
    Likes Received:
    183
    Location:
    District of Columbia
    One approach that sort of captures retired beers (and the historical perspective I'm interested in) would be to have Snapshot Top 250 lists. That is, have a (e.g.) 2011 Top 250 that is only calculated using reviews posted prior to Dec 31, 2011 and none after. Tally it the same way the Top 100 was prior to today to capture some flavor of the week beers at that time.

    I realize this would probably create more clutter than you want in your Top lists and additional server/db load, but I think something similar would be appreciated.
     
  6. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    Anyway, it's obvious that you simply don't agree with this particular change as you're hung up on a single beer taking a temporary hit. Again, if Kaggen is indeed that good it'll rise again.

    Beer. Fun.
     
    Onenote81, fredmugs and woosterbill like this.
  7. kevanb

    Beer Trader

    Joined:
    Apr 4, 2011
    Posts:
    1,097
    Likes Received:
    4,237
    Location:
    Illinois
    Doesn't it currently seem like beers that a LOT people say are above average (I.E 90 Minute, Alpha King, etc) are now being placed significantly ahead of beers that have many people saying are world class (Bligh's, Oude Gueze Vintage, RareR, etc).

    RareR D.O.S has a 4.6 average review while Alpha King has a 4.3 average review, sounds to me like one is CLEARLY better than the other, yet sits nearly 40 slots below.

    Call me crazy, but I think the Popular - Beers of Fame lists have started to blend a little too much.
     
    gjahn123 and eawolff99 like this.
  8. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    Like another in this thread, you seem to be ignoring how the lists are being generated; scroll down to the bottom of the lists to find out or read some of the replies in this thread. And okay, you're a little crazy.
     
  9. kevanb

    Beer Trader

    Joined:
    Apr 4, 2011
    Posts:
    1,097
    Likes Received:
    4,237
    Location:
    Illinois
    I understand it completely, but perhaps the 100 minimum scores is too high of a threshold, it starts to put a much more significant emphasis on total reviews (which is the whole point of Beers of Fame right?) Perhaps 50 would be better? You said that if Kaggen! really is as good as it says, it would rise, but with the current new landscape, I'd disagree, it seems that for every 1 review of Kaggen! there are roughly 24 reviews of Alpha King. I'd argue that because of the infrequency of it's reviews that it would be punished, even though the next review will most likely be significantly higher than the average reviews of Alpha King.

    Wasn't the whole philosophy of the Popular list to reward beers that are world class but might not get the same national exposure as those that met the 1000, or now, 2000 review mark?
     
    gjahn123 and pmarlowe like this.
  10. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    No. It's not too high (especially for all of the reasons already given) and beers that are truly world-class, but have a lower number of ratings, are still being represented.
     
  11. pmarlowe

    Beer Trader

    Joined:
    Nov 27, 2010
    Posts:
    789
    Likes Received:
    682
    Location:
    District of Columbia
    It looks like the new list is dramatically favoring beers with a large number of reviews, which makes sense given how you're calculating the WR. Although I'm not exactly sure why it is weighted that way.
     
  12. kevanb

    Beer Trader

    Joined:
    Apr 4, 2011
    Posts:
    1,097
    Likes Received:
    4,237
    Location:
    Illinois

    Fair enough. I am definitely interested to see how the lists evolve over the coming weeks/months.
     
  13. sbrotschul

    Joined:
    Oct 8, 2011
    Posts:
    55
    Likes Received:
    71
    Location:
    New Jersey
    Thanks for doing to much for all of us. This is a fun/educational/interesting/maddening site, which helps me to find those beers worth finding (and lets me comment on them as I am able!).

    Keep up the great work!
     
  14. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    We've been using the same WR algorithm for over a decade now.
     
    cavedave and Duff27 like this.
  15. pmarlowe

    Beer Trader

    Joined:
    Nov 27, 2010
    Posts:
    789
    Likes Received:
    682
    Location:
    District of Columbia
    Sure. You're just shrinking the rAvg number towards something much smaller now that the average across all beers on the list is 3.76.
     
    gjahn123 likes this.
  16. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    The mean (C) moved from 3.67 to 3.76. We lost 12 beers that were under the 100 threshold, but a good number of beers that were on the former list are on the new one; though their positions on the list obviously shifted.
     
  17. hooliganlife

    Beer Trader

    Joined:
    Apr 12, 2007
    Posts:
    488
    Likes Received:
    359
    Location:
    Missouri
    shit, now i am only missing 4 of the top 50

    ISO, this is doable
     
    Rutager likes this.
  18. drabmuh

    Joined:
    Feb 7, 2004
    Posts:
    139
    Likes Received:
    134
    Location:
    Maryland

    I'm only missing four on the whole list. I'm with you. ISO
     
  19. PatriotsRule

    Joined:
    Sep 25, 2012
    Posts:
    543
    Likes Received:
    426
    Location:
    Massachusetts
    I like these changes, gives me a heck of a lot mo brews to look forward to. Eight out of 250 thus far tried... I guess kiwi rising doesn't have enough ratings but it really is world class
     
  20. yamar68

    Joined:
    Apr 1, 2011
    Posts:
    3,823
    Likes Received:
    19,565
    Location:
    Minnesota
    If you decided to put together a top retired beers list (which I think would be way cool), is there any way you'd be able to allow people to review them without affecting the beer's score?

    If that's at all doable, I think it would also be nice to have that as an option for all beer reviews - maybe a check box like 'do not influence overall beer score.' I think it'd be a great tool for reviewing older beers, infected vintages, et cetera... just as a mode of record keeping and reference for those who might be curious.
     
    kojevergas and hooliganlife like this.
  21. hooliganlife

    Beer Trader

    Joined:
    Apr 12, 2007
    Posts:
    488
    Likes Received:
    359
    Location:
    Missouri
    could not agree more
     
  22. cavedave

    Beer Trader

    Joined:
    Mar 12, 2009
    Posts:
    7,871
    Likes Received:
    43,421
    Location:
    New York
    This doesn't sound doable, but I have been wrong once before.
     
  23. hardy008

    Subscriber

    Joined:
    Jan 12, 2008
    Posts:
    940
    Likes Received:
    2,055
    Location:
    Minnesota
    Thanks for all of your hard work. The new lists look great.
     
  24. sherm1016

    Joined:
    Aug 10, 2009
    Posts:
    852
    Likes Received:
    676
    Location:
    Wisconsin
    For top retired beers, does the databases store the date the beer was retired? i.e. only include ratings that were entered prior to you retiring the beer.
     
  25. Thehuntmaster

    Joined:
    Sep 2, 2009
    Posts:
    686
    Likes Received:
    2,963
    Location:
    South Africa
    What about a beer that 200/200 people rate 5 being ranked below a beer that 1000/5000 people rate a 5, that also doesn't seem quite right ;)
     
  26. codysjb

    Beer Trader

    Joined:
    Jun 16, 2010
    Posts:
    419
    Likes Received:
    244
    Location:
    Prince Edward Island (Canada)
    Any chance you have a list of the 2,700 beers that were retired? Just curious.
     
  27. DelMontiac

    Joined:
    Oct 22, 2010
    Posts:
    1,673
    Likes Received:
    7,737
    Location:
    Oklahoma
    Looks good. Nice job.
     
  28. woosterbill

    Beer Trader

    Joined:
    Apr 6, 2009
    Posts:
    1,020
    Likes Received:
    1,455
    Location:
    Kentucky
    I suppose this could work, although I honestly don't see the problem with allowing new reviews of retired beers. Obviously it depends on a beer's style, but particularly when it comes to the kinds of brews that would be on a Top Retired list I would venture a guess that significant periods of prime drinkability occur post-retirement. I don't think that the brewer of a Barleywine or Geuze has any problem with drinkers enjoying and reviewing their beers for years, if not decades, after they might be "retired."

    Anyone who thinks that their retired Armand'4s are over the hill should feel free to send them my way.
     
    huadog and Thorpe429 like this.
  29. Thorpe429

    Beer Trader

    Joined:
    Aug 18, 2008
    Posts:
    420
    Likes Received:
    365
    Location:
    Illinois
    Exactly this. I'd be curious how many beers that clearly do not age well would make such a list; my guess is very few.
     
    huadog and woosterbill like this.
  30. UCLABrewN84

    Joined:
    Mar 18, 2010
    Posts:
    7,936
    Likes Received:
    7,933
    Location:
    California
    Awesome, especially the Pacific only listings.
     
  31. Douberd

    Joined:
    Jan 27, 2012
    Posts:
    50
    Likes Received:
    46
    Location:
    Netherlands
    Any chance we get more info on data such as beers added, photos added, different countries etc, as someone mentioned on this thread's first page?
     
  32. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    This is unrelated to this thread and has already been addressed numerous times in previous threads. Extended user data/stats will happen with the site redesign.
     
  33. BrownNut

    Joined:
    Jul 11, 2011
    Posts:
    248
    Likes Received:
    97
    Location:
    Florida
    Hey Todd, I'm curious about this, though not necessarily from the standpoint of lists - just of reviewing in general. From this comment and another previously, it sounds like you guys see vintage reviews as flies in the ointment (with "old beer" being a different and more problematic item in my book, like the year-old IPAs my redneck liquor store still has on shelves). But given how much people cellar beers here, it would seem that cellared beers are pretty legit items to review, and that there would be a market for those reviews. I'm wondering what your thoughts are on how reviews of prior releases of a given beer mesh with reviews and scoring of current releases.

    Also I'm wondering if it would make sense to have separate entries for successive years of seasonals or limited releases. I feel like sometimes I see this (e.g. Trader Joe's Vintage Ale - but maybe the formula changes and it's truly a different beer? I don't know.) and sometimes not (most beers). That way the 2009 [whatever] that was infected won't pollute the reviews from uninfected years. And a fresh Expedition wouldn't have to compete with a five year old Expedition, which are really two different beers when it comes down to the taste and experience of them. And people could get an idea of how well a given vintage is holding up years down the road. I know there have to be complications here and am wondering what the Bros' take on them is since you have to do the math and wrestle with different ways of displaying things. Thanks.
     
  34. bobv

    Beer Trader

    Joined:
    Feb 3, 2009
    Posts:
    1,183
    Likes Received:
    13,917
    Location:
    Vermont
    Best. Update. Ever.
     
  35. jacksback

    Joined:
    Jul 20, 2011
    Posts:
    498
    Likes Received:
    323
    Location:
    Massachusetts
    Haven't really looked at the lists yet, but damn.

    Twice in one thread, people suggest something that might be helpful (25 to 50) or note a bug (drop down too small to see). Todd fixes within minutes.

    Sure, it likely doesn't take a programming rocket scientist, but still, that's pretty cool.

    /thumbsup
     
    tehzachatak likes this.
  36. LiquidCourage

    Beer Trader

    Joined:
    Nov 1, 2012
    Posts:
    817
    Likes Received:
    2,410
    Location:
    Rhode Island
    For the longest time Kaggen! had just 89 reviews.. 89! How can just 89 people determine that the beer is rated in the top ten in the world? The only thing I like about ratebeer better than BA is that ratebeer lets you filter their top 50 list to have minimums.

    I agree with Todd, if the beer it is that good, it will rise.
     
  37. mudbug

    Joined:
    Mar 27, 2009
    Posts:
    1,829
    Likes Received:
    3,091
    Location:
    Oregon
    West Coasters. We heard you. We now have two separate lists: Northwest and Pacific. Enjoy.

    Thank You! Well done!
     
  38. eawolff99

    Beer Trader

    Joined:
    May 10, 2010
    Posts:
    458
    Likes Received:
    357
    Location:
    Minnesota
    I understand the minimum threshold of reviews / hads to rank a beer argument and I'm not saying that 20 reviews giving a beer a 5 should rocket it to #1, but Todd should work with statisticians on statistical significance thresholds for a beer review -- is 50 as a minimum the right number, is it 75, is it 100? I don't know, but having rating numbers drive the rankings doesn't make sense. Using Kaggen again as an example -- I have had it, and Abyss. It is a SIGNIFICANTLY better beer and for it to be 100+ rankings below Abyss invalidates the point of rankings. It's like Susan -- it's the #1 rated American IPA with 179 reviews, yet #194 on the list of top 250 beers. It doesn't make sense. There aren't 179 Hill Farmstead homers artificially inflating this score...at a certain point, statistical significance threshold has to be met and the # of ratings shouldn't be driving the score, the actual score should be driving the ranking because it's supposed to be based on taste, not # of people that have tried it. Whew. Ok, whatever, I'm done on the point...just doesn't make sense to me.
     
  39. Todd

    Todd Founder
    Staff Member Fest Crew Subscriber Beer Trader

    Joined:
    Aug 23, 1996
    Posts:
    6,980
    Likes Received:
    17,738
    Location:
    Colorado
    So basically you're saying you don't know what the min should be, neither do we, but if we paid a statistician to analyze our data and picked one you'd be okay with it. o_O We actually put a lot of thought and looked into the data before we picked 100. It wasn't an arbitrary decision, but rather one that you don't agree with/understand.

    Also, you're comparing two separate lists with your American IPA example. The Top 250 Beers min is 100, whereas the beer style is 10. This explains the differences in rankings.

    Finally, both the overall rating and number of ratings drive the rankings behind the lists. It's not necessarily one or the other. Oh, and "taste" ultimately does prevail.

    Hey look ... beer! I'm going to grab one.
     
  40. JulianB

    Joined:
    Feb 1, 2012
    Posts:
    1,020
    Likes Received:
    2,408
    Location:
    South Carolina
    This is actually a great example - to argue against the point you are making. The number 1 rated IPA should be whatever the hot new release is from Hill Farmstead? Really? The fact that it's not blasting up the Top 250 list is a good thing, it means it needs to sustain more than just the initial release hype to "earn" its ranking. If anything that kind of ranking is a good argument against raising the threshold for the style lists, 180 reviews/hads and it's ahead of stuff like Nelson, Duet, and Blind Pig?
     
    wingedeel, hardy008 and ant880 like this.
  • About Us

    Your go-to website for beer (since 1996), publishers of BeerAdvocate magazine (since 2006) and hosts of world-class beer events (since 2003). Respect Beer.
  • Return of the Belgian Beer Fest

    BeerAdvocate Brings its All-Belgian Fest to Portland, Maine on September 17, 2016. Tickets are on sale now.

    Learn More
  • Get the Mag

    Become a BeerAdvocate magazine print subscriber today.

    Subscribe