Updates to the BA Score, Rankings, and Top Rated Beers

Discussion in 'BeerAdvocate Talk' started by Todd, Jul 22, 2019.

  1. Todd

    Todd Founder (5,982) Aug 23, 1996 California
    Staff Moderator Fest Crew Society

    Yeah. I know what you're thinking. Not again! Why?!

    It was time for some internal disruption.
    • Unfortunately we've seen a drop in the industry using our scores since moving to 1-5 point system. We’d like to change that.
    • We know that many users miss the old 100 point system. In fact, the last poll we took indicated the most users preferred it. Perhaps we should've of listened, but we've learned a lot.
    • The 100 point system feels more BA-like and supports other projects that are in the works.
    • The Top 250 and regional lists became pointless and boring for most; same styles, from the same breweries.
    • We want to place more focus on the beer’s style and style education in general.
    Thanks to feedback from the industry, recent threads, and Society members, I stoked to announce the following updates.

    BeerAdvocate (BA) Score
    Unlike past versions, the math isn’t simple on the new BA Score.
    1. First, whenever a beer is reviewed we obtain the truncated mean. This means we examine all user scores for the beer, but then toss out an equal percentage of outliers on both ends.
    2. Next, we then obtain the beer’s weighted rank (WR) using the truncated mean. This entails running it through a custom Bayesian formula that takes the beer’s style into consideration; specifically the average and average number of ratings across all active beers within the style. This means that a beer is no longer pitted against all beers, but rather all active beers within its style.
    3. Finally, we apply the 100 point score using the beer’s weighted rank.
    A beer must have 10 or more ratings to be assigned a BA Score. And the BA Score is the only score displayed. You won’t find the beer’s average, truncated mean, or weighted rank. (read the update)

    Rankings
    All beers that meet our criteria (WR + activity) are now ranked within their listed style. Rankings are shown on beer pages and used to generate top rated lists for styles. Style data and rankings are updated daily.

    Top Rated Beers
    As we’re no longer comparing a beer against all other beers, we’ve opted to retire the old Top 250 beers, regional lists, etc. Instead, we’ll be focusing on where a beer ranks within its style and the new Top Rated Beers page reflects this new focus. We’ll eventually explore adding more interesting and useful lists, too. (read the update)

    Other than some cosmetic tweaks, that’s it for now. And @Mike is working to support these updates in the app, too.

    I’m currently rolling out the updates, so please be patient. Issues? Let me know. Otherwise I look forward to your feedback.

    Cheers!

    Todd
     
  2. JerzDevl2000

    JerzDevl2000 Poo-Bah (2,891) Oct 7, 2005 New Jersey

    I like it - the pages have a cleaner look and the information is easier to see. I like how more emphasis is being place on styles that taking a beer and comparing it to everything across the board. Keep up the good work!
     
  3. paulish

    paulish Poo-Bah (1,849) Feb 2, 2014 New York
    Society Trader

    Bad news:slight_frown:
     
    Lucoli, VoxRationis and fthegiants like this.
  4. bbtkd

    bbtkd Poo-Bah (2,745) Sep 20, 2015 South Dakota
    Society Trader

    Specifically?
     
  5. Futbolmanby

    Futbolmanby Crusader (701) Jul 7, 2015 Michigan
    Trader

    Should keep the top 250 as a separate list along with the beer of fame list... it was a cool bucket list type thing for us old timers
     
  6. DIM

    DIM Poo-Bah (2,717) Sep 28, 2006 Pennsylvania
    Society

    Looks great!
     
    Viaduck and Todd like this.
  7. superspak

    superspak Poo-Bah (32,407) May 5, 2010 Michigan
    Moderator Society Trader

    Can we still have an average score listed under the numeric score? I have no idea how the new score correlates with the number, at least show both. There are going to be a bajillon beers that have no score for my reviews too, since there are not enough ratings.

    Love that the top rated beers are by style now though.
     
    #7 superspak, Jul 22, 2019
    Last edited: Jul 22, 2019
  8. paulish

    paulish Poo-Bah (1,849) Feb 2, 2014 New York
    Society Trader

    It's look the same. May be latter we will see a changes.
     
  9. DIM

    DIM Poo-Bah (2,717) Sep 28, 2006 Pennsylvania
    Society

    No, look up any beer.
     
  10. DIM

    DIM Poo-Bah (2,717) Sep 28, 2006 Pennsylvania
    Society

    Todd likes this.
  11. mig100

    mig100 Meyvn (1,378) Aug 3, 2014 Texas
    Society Trader

    I agree with this. Love top beers by style now, but the top 250 lists were a really great feature and should be kept.
     
  12. jvgoor3786

    jvgoor3786 Poo-Bah (1,700) May 28, 2015 Arkansas
    Society Trader

    Pages look nice and clean. I appreciate the reasons for the change. I will also miss the Beers of Fame and Top 250. Plus, I really like to know the average rating, since we rate 1-5, not 1-100.
     
    FBarber, Todd, von_kaiser and 9 others like this.
  13. AZBeerDude72

    AZBeerDude72 Poo-Bah (1,794) Jun 10, 2016 Arizona
    Society Trader

    Not sure I like the top beers list. I see a beer with 1000 reviews rated as #1 but one with 13,000+ reviews is rated #11. To me this is flawed, the formula needs to take into consideration number of reviews. It is pretty clear a beer like KBS should be rated higher. When you take into consideration the number of high reviews it is proof the beer is world class. To me 1000 reviews does not indicate a beer is great.

    Look at KBBS, #1 Imperial Stout with under 800 reviews? That is better than KBS? I really think the formula needs to weight in the number of reviews in order to rank a top beer.


    Double Shot | Tree House Brewing Company

    Wants Gots

    Reviews: 105 | Ratings: 1,078

    #1 ranked American Stout | 7.60% ABV

    KBS (Kentucky Breakfast Stout) | Founders Brewing Company

    Wants Gots

    Reviews: 3,472 | Ratings: 13,664
    #11 ranked American Imperial Stout | 12.30% ABV

    Kentucky Brunch Brand Stout | Toppling Goliath Brewing Company

    Wants Gots

    Reviews: 143 | Ratings: 767
    #1 ranked American Imperial Stout | 12.00% ABV
     
    #13 AZBeerDude72, Jul 22, 2019
    Last edited: Jul 22, 2019
    JediMasterLenin and paulish like this.
  14. tone77

    tone77 Poo-Bah (5,661) May 20, 2009 Pennsylvania

    I am already missing the top beers by state and country lists. This was a valuable tool when traveling to unfamiliar states/countries, giving me an idea of what beers to keep an eye out for.
     
  15. stortore

    stortore Poo-Bah (2,455) May 31, 2011 Illinois
    Society

    Totally agree. I have no way to compare my rating to the numeric score.
     
  16. mig100

    mig100 Meyvn (1,378) Aug 3, 2014 Texas
    Society Trader

    Agree 100% with both of these points also. These features made BA stand-out vs. the other beer rating sites/apps.
     
    ewpass, jakecattleco, Todd and 7 others like this.
  17. paulish

    paulish Poo-Bah (1,849) Feb 2, 2014 New York
    Society Trader

    Now I have only one option (app) for hunting a beer when I traveling. One week ago I had а two, including BA.
     
    MJSFS and tone77 like this.
  18. GratefulBeerGuy

    GratefulBeerGuy Poo-Bah (2,920) May 20, 2006 New Hampshire
    Society Trader

    100 point scale=thank you!!
     
  19. RaulMondesi

    RaulMondesi Poo-Bah (2,089) Dec 11, 2006 California
    Trader

    Speaking for the numerically inept, I am happy to see the 100 point scale back!
     
  20. Mebuzzard

    Mebuzzard Poo-Bah (3,555) May 19, 2005 Colorado
    Trader

    Sure are a lot of 100s for the NE IPA style.
     
  21. cryptichead

    cryptichead Meyvn (1,387) Jul 3, 2014 Illinois
    Society Trader

    1. Would definitely like to see the straight 1-5 average score on each page in addition to the BA score
    2. What happened to the 'View' link to take you to your own review/rating with the date?
    3. Top 250 and state pages would be missed and would be nice to have them back

    Otherwise, looking good
     
    Todd, stortore and Snowcrash000 like this.
  22. cryptichead

    cryptichead Meyvn (1,387) Jul 3, 2014 Illinois
    Society Trader

    4. Our own rating seems to be lost on the page and is not as prominent as it used to be (now it's a small black font number, used to be a bigger font red number)
     
    Todd and REVZEB like this.
  23. Phoodcritic

    Phoodcritic Zealot (579) Jul 3, 2014 Michigan
    Trader

    I like the new focus on style. I have two suggestions regarding the top beers by style:
    1. I've always felt that the adjectives "outstanding" and "exceptional" were virtually synonymous. Under the new format, the score range 85-89 seems a bit too low to suggest that a beer is truly exceptional. Might you replace "exceptional" with another word, perhaps even "excellent"?
    2. I think the "Top Rated Beers" page would be cleaner in appearance if the styles were displayed in bold (instead of having each top beer in the bold font).
    Cheers!
     
  24. FatBoyGotSwagger

    FatBoyGotSwagger Meyvn (1,366) Apr 4, 2009 Pennsylvania

    Good change, just need to work on getting every beer in it's correct style unless you are broadly expanding styles

    2 quick outliers would be Elliot Ness as a dark lager

    And Nugget Nectar as an American strong ale. There are probably a lot more but keep up the good work the top 250 became a joke.

    You could still keep the beers of Fame list for reference because that was a good one.
     
    Todd, mdaschaf and officerbill like this.
  25. Chuckdiesel24

    Chuckdiesel24 Meyvn (1,136) Jul 6, 2016 Illinois
    Trader

    Agree wholeheartedly with the above.

    The BA Top 250 list was a very cool thing that made this site stand out. I agree that there had become an element of too much of the same styles in the list, but honestly that had a lot to do with the underlying formula which allowed way too much weight to beers without a lot of checkins.

    The local threads were very helpful when traveling. The styles aren't so much because certain styles are dominated by regions. I'm leaving for vacation in two days; I'll be in West Virginia for a night and Delaware for a week and was thinking "I hope they get that change finished so those lists get back up." I guess I was right on the first part, wrong on the second.

    I do like emphasizing style more, but I've got to say - if people are using the other lists, why not still have both?

    I'd also say, some of the more popular lists still have the same issues. In today's world where there are 7,000 US breweries - this list shows one brewery making up half of the top 30 NE IPAs and 4 breweries making up 80% of the top 50. I think the issues were more about the formula being subject to how much more popular this site is in certain populations.

    Just my 0.02....

    Oh, and lastly. Maine Dinner is not a NE IPA!!!!!
     
    Todd, von_kaiser, hopley and 9 others like this.
  26. TrojanRB

    TrojanRB Meyvn (1,471) Jul 27, 2013 California
    Society Trader

    Generally positive, but I miss the Top-250 and Beers of Fame. It was somewhat of a goal to check off as many as possible.
     
    FBarber, Todd, von_kaiser and 12 others like this.
  27. mig100

    mig100 Meyvn (1,378) Aug 3, 2014 Texas
    Society Trader

    Exactly, why take away features? Add new ones, like the top beers by style... but no reason to eliminate old ones.
     
    hopley, spersichilli, MJSFS and 8 others like this.
  28. Ajlvt

    Ajlvt Zealot (505) Jun 7, 2011 Washington
    Trader

    rough update. please bring back the top 250 and a way to see the average rating. adding a new weighted score out of 100 is no compensation for the things that were taken away
     
  29. JLK7299

    JLK7299 Meyvn (1,075) Apr 21, 2016 North Carolina
    Society Trader

    Style rankings have always existed so I’m a little confused. 90% of my reason for being on this site is the top 250 and my quest to complete it, please restore it!
     
    sulldaddy, ragnar3, hopley and 9 others like this.
  30. von_kaiser

    von_kaiser Defender (699) Feb 19, 2014 Connecticut
    Trader

    put me in the category of bring back the top 250, top beer by state/country, beers of fame, etc. those lists make this site fun and enjoyable as well being helpful.
     
    Radome, sulldaddy, ragnar3 and 17 others like this.
  31. AlcahueteJ

    AlcahueteJ Meyvn (1,089) Dec 4, 2004 Massachusetts

    Fantastic.

    I've been waiting for something like this for years.

    I still did like seeing how every beer compares against one another, just to see what people like overall, disregarding style.

    But still, great job, I love Pilsners not being penalized for...well, being Pilsners.
     
  32. JLK7299

    JLK7299 Meyvn (1,075) Apr 21, 2016 North Carolina
    Society Trader

    More than a goal. I was obsessed. Without the top 250, I’ll honestly rarely visit this site never mind be a Premium Member.
     
  33. AlcahueteJ

    AlcahueteJ Meyvn (1,089) Dec 4, 2004 Massachusetts

    I've had Double Shot, and in my opinion it's FAR superior to KBS.

    While I do agree, the number of ratings should be taken into account, we also shouldn't reward a beer if better versions of that style have been brewed in the past few years.

    A lot of those higher reviews for KBS are likely because it used to be more rare, and there was less barrel-aged beers when it first was brewed.

    Not because it's necessarily better than Kentucky Brunch Brand Stout.

    I think it had way more to do with people's preferences for New England IPAs and pastry stouts.

    I'm not sure about this either.

    There was a time before Tree House, Trillium, Hill Farmstead, Alchemist...and Russian River did fairly well on this list, as well as Founders, Westvleteren...etc.
     
  34. Harrison8

    Harrison8 Poo-Bah (3,608) Dec 6, 2015 Missouri
    Society Trader

    Is there site data supporting getting rid of the Top 250, Beers of Fame, etc.? I would think these would be high traffic areas of the site, netting the site more ad revenue for frequent, non-Society members.

    In regards to the rest of the changes - I like them, but that's not necessarily a good thing. I'm a bit of an outlier in the local craft beer circles in that I love 'ticking' odd styles more than others. Emphasis on the styles, while helpful for me, carries much less relevance for those I know in the local craft beer scene.

    P.S. If we could get these top style lists on the mobile app, that would be fantastic. Although any sort of data compilation lists on mobile would be great to help guide purchases while traveling.
     
    von_kaiser, Eman17, MJSFS and 6 others like this.
  35. Hopelessly0

    Hopelessly0 Initiate (123) May 13, 2019 California
    Trader

    I actually liked the Most Popular 250 list as well. I was going down the list to try them all, even if it's lowly rated. I think bringing that back would be nice.
     
    von_kaiser, mdaschaf, MJSFS and 4 others like this.
  36. Hayley_86

    Hayley_86 Poo-Bah (2,843) Jan 1, 2014 Netherlands
    Moderator Society Trader

    Change can definitely be good sometimes.

    So switching back to the 100 point system can be a good change. I don’t prefer it but I also don’t mind it.

    And I can understand having a stronger focus on styles within the spectrum. So having prominent Top lists of beers within a style gives us a new way of looking at the rankings. And more insight and ways to look at things is usually a good thing. It is interesting to know that the beer I am drinking is the #1181 American Imperial Stout on BA.

    The regional lists absolutely have their flaws. I live in the Netherlands and the top 100 for here in some ways honestly represents the beer scene here and in some ways definitely doesn’t. Broadly available beers outrank lesser available beers even though the quality of the beer doesn’t justify it. So I understand giving these lists less prominence on the website.


    An evaluation of the ranking formula is a good thing as long as it doesn’t happen to often. A beer should have a minimum number of ratings before it counts, otherwise the lists are to easily manipulated. But now the beers get a rating solely within their style. I don’t rate beers purely within their style and I doubt everybody else does. And with the new rating it will not tell me whether a 72 scoring Berliner Weisse is considered a better beer by the BA community then a 92 scoring Russian Kvass. But that is something I would like to know.

    I found BeerAdvocate years ago because I was looking for the answer to what is the best beer in the world. The top 250 has been the most interesting page on this website ever since for me. And so I am sad that the list is taken down. Adding new things will improve BA but removing things rarely makes it better. And also the most rated list and the beers of fame were fun lists to look at and challenge myself to find those beers. I highly doubt I will look to find all the top 100 German Altbier

    I understand no list is perfect and the top 250 will always spark debate. But not having that list anymore seems to me to be the worst option. So I hope it will be brought back.
     
  37. seakayak

    seakayak Defender (668) May 20, 2007 Massachusetts
    Society

    @Todd, kudos to you and the staff for the welcome updates to the site. Lots of work, I'm sure.

    Something to consider - ratings within style are only valid IF the styles are defined correctly. Trillium's Double Dry Hopped Fort Point Pale Ale, plus their Galaxy, and Mosaic, and many others, are still incorrectly dominating the Top Rated Pale Ales because these obvious NEIPA's were incorrectly assigned to the Pale Ales in the first place, apparently because the brewer misidentified it this way. Also, because BA has no NE-style pale ale or NE-IIPA categories (plus many others frequently discussed here), this problem will persist, and it will diminish the value of the new rating-within-style listings.

    Maybe fixing style-definition should have happened before the change to rating-within-style. Perhaps that can be a focus of future updates and improvements.
     
    Radome, meefmoff, jrc1093 and 7 others like this.
  38. AZBeerDude72

    AZBeerDude72 Poo-Bah (1,794) Jun 10, 2016 Arizona
    Society Trader

    Totally agree. The hard part with all this is using all the factors involved. Not sure it can be done but I really feel number of reviews does play a big part. A lot of high ranked beers have very low review numbers so are they really great or is it hype and brewery fans rating them high out of loyalty? Not sure how this could be solved I know it has come up before.
    Cheers
     
    Raime, AlcahueteJ and kuhndog like this.
  39. fthegiants

    fthegiants Poo-Bah (1,571) Nov 20, 2013 California
    Society Trader

    Not Happy.... Top 250 and Top New Beers was what i loved most about this site. (even with their flaws)
     
  40. AZBeerDude72

    AZBeerDude72 Poo-Bah (1,794) Jun 10, 2016 Arizona
    Society Trader

    Agree with the lists, I understand change happens but I think we can have both. The lists you mention should be available in some form for people to use, seems like so many do want them and viewed them. I hope there is a way to have both new and old, cheers.
     
    JLK7299 and Hayley_86 like this.