Proposed Updates to Top Rated Beer Lists

Blog Discussion in 'BeerAdvocate Talk' started by Todd, Dec 28, 2021.

Thread Status:
Not open for further replies.
  1. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Check out: End of Year Site Updates

    ###

    In the past, I'd make a massive update that would impact our top rated beer lists, collect feedback, and make adjustments. For this update, I'd like to discuss one update that's already occurred and propose two major updates for discussion before anything is applied.

    Occurred: The Raw Average Is Back

    In the before times, we switched from using the beer's average to a truncated (trimmed) mean, wherein we were essentially discarding any ratings that were outliers equally on both ends of the 1-5 rating scale. While it seemed like good practice at the time, I don't like the idea of discarding opinions on a beer simply because they're numerical outliers. I'd rather let the raw average come to play, and then let the weighted rank sort it out.

    This update has already been applied.

    Proposed: Consider Active Beers Only

    Only consider "Active" beers (not retired and rated within the last two years), when ranking beers. This will no doubt help to remove beers that are no longer available/produced from camping the lists, but it'll significantly limit our pool of ranked beers. Why? See below.

    Proposed: Lower the Minimum Threshold
    Since basically forever, the minimum number of ratings that a beer needed to be assigned a BA Score and to be considered for top rated beer lists (ranked) has been 10. Today, there are over 9,000 breweries in the US alone releasing hundreds of thousands of beers a year, we've seen a decline in ratings over the years, less new beers being added in recent years, and most beers being entered today are one-offs (brewed once and retired after a year of being entered on the site).

    Given all of this, it would make sense to lower the minimum to 5 ratings. This would immediately double the pool of ranked beers, give more beers a BA Score, and help users discover new beers. If we don't, it's going to increasingly become more difficult (or impossible) for new beers to ever make the lists unless they're regular offerings that gain wider distribution. Even then, it could take a very long time.

    Note: I understand that some of you who chase/tick the Top 250 and whatnot will oppose any significant changes that would thwart your efforts, but I'm looking for feedback that goes beyond that and considers the bigger picture.

    Thanks in advance for your thoughts.
     
  2. Sheppard

    Sheppard Grand Pooh-Bah (3,516) Mar 16, 2013 Massachusetts
    Pooh-Bah Trader

    I am not a Top 250 ticker and I am someone looking out for new local options that stand out, so these changes would certainly be welcome for me. I've mentioned this in the Top State Beers thread, but it's hard to discover new breweries (unless you just go to whatever is out there) since the lists are already congested or even dominated by established breweries. Hopefully, this evens the playing field a little and allows for someone who does something really well to breakthrough. Based off my criteria, I welcome these changes.
     
    Edgewise, DrOfGolf, scott451 and 18 others like this.
  3. beergoot

    beergoot Grand High Pooh-Bah (9,310) Oct 11, 2010 Colorado
    Mod Team BA4LYFE Society Pooh-Bah Trader

    I like your ideas for the raw average plus listing only using active beers for the top rankings. The minimum rating, I can take that or leave it. I suppose it's worth trying out and seeing how it impacts ratings.

    I imagine the beers of fame still will include retired beers, correct?
     
  4. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    It's never (intentionally) included retired beers.
     
  5. Harrison8

    Harrison8 Grand High Pooh-Bah (6,285) Dec 6, 2015 Missouri
    Pooh-Bah Trader

    Per the 'active' - I'd say allow active & inactive to be apart of the lists. If anything, beers landing in 'inactive' can be more easily flagged by BA users to 'retire' them. Keeping this process manual means there could be some more evidence for when a beer is truly retired vs. simply not active on BA.

    Personally, I'm less than thrilled with bumping the BA score threshold down to five, but ultimately, I think it's the right move. Even the more frequently rated/reviewed beers from breweries local to me are only hitting ≤ 8 ratings at best. That's after several re-releases.

    It could create more of a mess than anything, but anyway to supplement beer listings under 10 ratings/reviews with an averaged Untappd score? Probably a PITA to link them all, with the different naming conventions and all that, but wanted to throw it out for consideration.
     
    Edgewise, ChicagoJ, Whyteboar and 8 others like this.
  6. JohnnyHopps

    JohnnyHopps Grand Pooh-Bah (3,380) Jun 15, 2010 Indiana
    Pooh-Bah Trader

    I’m a little concerned about a one-off that gets (only) five reviews getting an exaggerated placement in the top 250. 5 reviews at a popular a brewery is not a lot. In the long run, I don’t think this helps more obscure breweries get recognized.
     
    ADTaber, defunksta, cjgiant and 13 others like this.
  7. zid

    zid Grand Pooh-Bah (3,132) Feb 15, 2010 New York
    BA4LYFE Society Pooh-Bah Trader

    Personally, from a conceptual standpoint, I think a minimum of 10 ratings for a beer to be on a communally created global top beers list is too low rather than too high. If 10 is too high from a practical standpoint, then I’d question the purpose of even having such a conceptually compromised consumer driven list.
     
    ADTaber, hopley, wspscott and 19 others like this.
  8. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    I've been a big proponent for a unified BeerAdvocate/Untappd brewery/beer database, and I think we'll eventually get there, but it's a massive project and there are bigger priorities at the moment.
     
    Edgewise, Junior, Mikexw and 15 others like this.
  9. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Just like you don't see any beers with 10 ratings in the Top 250, you're not going to see beers with 5 ratings in the Top 250. You'll see them in some regional and styles lists though.
     
    Junior, Mikexw, ChicagoJ and 12 others like this.
  10. UrbanCaveman

    UrbanCaveman Pooh-Bah (1,866) Sep 30, 2014 Ohio
    Pooh-Bah Trader

    Regarding the "active beers only" idea, I've run across several beers marked as retired that have apparently been pulled out of mothball status for re-release, the most recent being Listermann's Maple Chickow. How is retirement status currently determined, for beers that aren't one-offs but also aren't necessarily on a regular schedule?
     
  11. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    The minimum can and should be flexible based on activity. You see this with IMDB (we use a similar weighted rank system). Though they no longer disclose their criteria, it's changed over time.
     
  12. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Beers are retired manually (by a report from a user or brewer) or automatically retired after one year if the beer was entered as a one-off.
     
  13. paulish

    paulish Grand Pooh-Bah (3,264) Feb 2, 2014 New York
    Society Pooh-Bah Trader

    Sadly... Welcome 250 kinds of Pastry Stouts.
    My suggest - The beer with 5 ratings will be to able to include in BA rankings but not more one year. If the beer cannot collect more than 10 for year the beer will remove from any BA rankings.
     
    Sammy, rodbeermunch and JohnnyHopps like this.
  14. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

  15. zid

    zid Grand Pooh-Bah (3,132) Feb 15, 2010 New York
    BA4LYFE Society Pooh-Bah Trader

    Perhaps you should consider modifying this with the addition of “inactive.”
     
    Beer_Economicus likes this.
  16. PapaGoose03

    PapaGoose03 Grand High Pooh-Bah (6,057) May 30, 2005 Michigan
    BA4LYFE Society Pooh-Bah

    A little bit of a tangent here, but a long time ago there was discussion about some sort of 'handicap' factor to be applied to less popular styles that would help great beers in those styles reach the 'top' lists. Did that ever happen, or should it be part of the discussion in this thread?
     
  17. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    As in auto-retire them? If so, that assumes that the beer is no longer being brewed. But running with this for a moment, we could merge inactive and retired into simply inactive. One less tab, less confusing, and an inactive beer could easily become active again with a simple report if needed.
     
    laketang, DrOfGolf, scott451 and 13 others like this.
  18. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    No. This never happened. Big can of worms, IMO. But feel free to start a new thread.
     
  19. Coronaeus

    Coronaeus Grand Pooh-Bah (3,744) Apr 21, 2014 Canada (ON)
    Society Pooh-Bah Trader

    While I understand the hesitation from some regarding lowering the necessary ratings to 5, this proposal might actually help get the Canadian list (as well as others I’d presume) move out of 2014.

    Here, we suffer from too few users rating or reviewing beers. I’m not sure how that will ever change. We also see mediocre breweries that distribute to the US take far too many spots on the list. The breweries that most local beer drinkers would mention as being ‘among the best’ are almost completely absent from the list. I don’t know if the proposed changes will help make the list more reflective of the current beer scene here, but they might.

    If you take one of the most popular styles, NEIPA, you’ll find next to none of the generally acknowledged best Canadian makers of the style on the list currently. Just from Ontario and Quebec alone, Messorem Bracitorium, Bas Canada, Wood Brothers, Badlands, Barncat, Third Moon should dominate the list. They don’t. This would be like Other Half, Tree House, Monkish etc., not making a top list in the US, because Celis or Pete’s Wicked Ale are still holding down spots.


    One other thought. For the tickers, perhaps an annual archived Top 250 list would be a nice thing to have easy access to.
     
  20. PapaGoose03

    PapaGoose03 Grand High Pooh-Bah (6,057) May 30, 2005 Michigan
    BA4LYFE Society Pooh-Bah

    I can see it being a big can of worms to create factors to use, and you don't need that right now. Anyone who wants to see top beers in a particular style can go to the style page and find the top beers there.
     
Thread Status:
Not open for further replies.