New BA Score, Beer Ranking & More Updates

Discussion in 'BeerAdvocate Talk' started by Todd, Sep 19, 2017.

Thread Status:
Not open for further replies.
  1. Oktoberfiesta

    Oktoberfiesta Initiate (0) Nov 16, 2013 New Mexico

    Another local example, this time from the same brewery.

    3.98 rated IPA vs. a 3.77 rated IPA
    86% liked vs. 63%. For a beer that is rated only 5% better, there appears to be a 4 times multiplier in enjoyment. At least thats the perception that they are now giving by BOLDING the % liked.

    Each 0.10 difference is amounting to some 10% change in % liked, for those beers within that 3.75 threshold. To me that feels like a flawed threshold. It's just said a totally respectable beer like SA Latitude 48 now sits at 42.5% liked (r score of 3.56)

    Another example. Ranger IPA vs. Lagunitas IPA. Ranger is a 3.77 rated beer and 65% liked. Lagunitas IPA is a 3.89 rated beer and 76% liked. It's not like this for every beer. But every 0.01 change in the R, we're seeing the % like increased by 1% point.

    There's too much variance around 3.75. So much so that when you look at the top 95%- 99.5% liked beers, you really can't distinguish between them. We're seeing some beers in the 4.35 rated range coming in nearly the same % liked as a 4.60 rated product. If we compared the middle to the top end, we should see a 25% difference in % liked. A 97% enjoyed 4.35 beer should compare to a 4.60 122% enjoyed beer. Obviously that's above the 100% max. But it just shows to me anyway that something isn't right.

    When it comes down to it, we're seeing 3.52 rated beers coming in at 42% liked, then 0.20 rated higher beers coming in at 63% liked, and then 0.20 beers above that, 85% range. At a certain point, your 4.60 perfect beers should be ABOVE PERFECT. That's what I'm getting out of it.

    I like the muddling of the numbers on the top of the spectrum where 0.25 difference in scores only amounts to 2% difference in enjoyment. That part I can understand. But the 3.75 marker is just brutalizing so many beers that the % liked becomes a joke

    Ill stop beating the dead horse now.
    #121 Oktoberfiesta, Sep 23, 2017
    Last edited: Sep 23, 2017
    ManapuaMan, JFresh21, Lucular and 9 others like this.
  2. BrewNoob1

    BrewNoob1 Initiate (0) Jan 8, 2015 Minnesota

    Arcadia Brewing Company Bourbon Barrel Shipwreck Porter. Thanks for the assistance.
  3. dbrauneis

    dbrauneis Grand High Pooh-Bah (7,470) Dec 8, 2007 North Carolina
    Super Mod Pooh-Bah Society Trader

    That one has been reported and will be fixed in the not too distant future - there is a linked beer list between the two Arcadia locations, one of which closed recently which is causing the issue.

    @Todd I sent a note about unlinking them and moving active entries to the open location. Any ETA?
  4. Lone_Freighter

    Lone_Freighter Initiate (0) Jun 4, 2017 Vermont

    and to take your thought one step further - will this encourage more users to rate to style?

    makes me wonder....

    but a guy can hope....
    VABA likes this.
  5. TongoRad

    TongoRad Grand Pooh-Bah (3,826) Jun 3, 2004 New Jersey
    Pooh-Bah Society Trader

    Nice! Thanks for doing a lot of legwork on this.
  6. donspublic

    donspublic Grand Pooh-Bah (3,356) Aug 4, 2014 Texas
    Pooh-Bah Society Trader

    I doubt it. I saw someone rating some beer of a style that was supposed to have little to none hop profile and he dinged the hell out of it because of this. Not sure that will ever work it's way out. I have been going back tru the beers I had when I first started drinking good beer and trying to correct the dumbass things that I did. It is what it is, I can deal with the %, because I am always going to look at the number rating and the reviews, but google grabbing that % and throwing it up on the search page is killer.

    Off topic, but regarding Google. If you have Android Auto you can ask Google about a beer and she will read you the BA Notes section. Cracked me up. Kept me busy from work all the way home the other day.
  7. Lone_Freighter

    Lone_Freighter Initiate (0) Jun 4, 2017 Vermont

    Yup, guess that was a pipedream, oh phooeeey.

    Wait a minute, was this backlash because the Bro's changed the way the scores are done? Sounds pretty immature on that person's part. There's the "report abuse" button on the bottom left of people's reviews, ya know.

    But still my hat goes off to you for going back through your beers, that's a respectful thing yet possibly a daunting task to do something like that.

    And that's pretty cool thing that Google does! I have an Android, I'm gonna have to check this out. Thanks man!


    sidenote: and I'm not asking to be lynched on BA by anyone as I may be one of the few that doesn't think negatively or positively about the scoring changes yet. There's pros and cons and well, I'm just going to go with the flow for the most part to continue to see these things to fruition. There's always growing pains in changes and I'm not disrespecting anyone who has been beer reviewing longer than me that may think differently.
  8. donspublic

    donspublic Grand Pooh-Bah (3,356) Aug 4, 2014 Texas
    Pooh-Bah Society Trader

    No, while back I saw that, just a random review I was reading while deciding to purchase the beer. I think the thinking is, which can be problematic, that you simply rate a beer how you like it, not how well the beer is brewed to style (tons of threads debating this). That was the review that clicked that I needed to be more careful with how I review a beer. If I tried something and didn't like it, I needed to figure out if it was me or if that is the way the style was intended to be. As a matter of fact in looking at my rating for Pilsner Urquell I slid that puppy in there right above the line at 3.77. I had that about 1 year into my current beer craze. Didn't appreciate a pilsner the way I do now. But also not sure what shape that beer was in when I had it. Having drank a ton of it on tap when I was in Poland, I should have updated my review, but didn't (there was 1 place there that had only it on tap and poured it like 3 different ways, various levels of head).
    Lone_Freighter likes this.
  9. detgfrsh

    detgfrsh Savant (1,161) Jun 20, 2014 Texas

    Completely agree. How about another score that shows where a beer ranks within its style? A couple ways to do it:
    El Sully (a good example, it has a low 'like' rating but ranks well among AALs)
    Avg: 3.46/5 (style avg: 3.00)
    90th percentile in American Adjunct Lagers
    cryptichead likes this.
  10. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    Just to reinforce what @donspublic has said, even though I've not seen the particular review he mentiond, it's not likely to be backlash. I've been seeing such reviews for "lo these many years" now. I've decided it's an endemic problem for a site like this where there's no "entrance exam" or "training course" to qualify anyone as being a rater/reviewer.

    Many folks aren't looking to learn and so don't bother with the Beer 101 information and just start using the numbers, etc. So reviewing while "keeping style in mind" is something a lot of folks do try to do, others....

    This is one reason I only pay attention to reviewers I trust when making up a shopping list of stuff I think I might like to try. (The other being that I'm cheap and don't want to be spending money on beer I won't enjoy. :wink: )
    MNAle, donspublic, TongoRad and 2 others like this.
  11. Lone_Freighter

    Lone_Freighter Initiate (0) Jun 4, 2017 Vermont

    I may have been confused initially by what Donspublic was saying but I get it now.

    And yes, my guess the ratio of cicerones to regular beer drinkers is probably so far imbalanced towards the regular beer drinkers that you have a valid point in understanding that so many people most likely rate to personal preference in the beginning (we all have to start somewhere, right?). But there's probably also a fair amount of people that continue on to rate to personal preference too.

    I'd like to think that as people grow in their understanding of beer styles that many would want to come back to beers that they didn't understand at one time to try again to see if they like it the second time. Makes me wanna go back and see my own reviews to see what I can do better.
    VABA and drtth like this.
  12. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania

    If I'm having a beer I first reviewed "back in the day," I'll often take a look through the review to see if I want to update it or add to it, but otherwise ignore that urge and satisfy myself by concentrating on continuing to develop my understanding so that I can convince myself that I've been improving. :sunglasses:
    Squire and Lone_Freighter like this.
  13. Sabtos

    Sabtos Grand High Pooh-Bah (6,984) Dec 15, 2015 Ohio
    Pooh-Bah Society Trader

    Same same :beers:
    VABA likes this.
  14. Ranbot

    Ranbot Pooh-Bah (2,327) Nov 27, 2006 Pennsylvania

    I agree with all of this. No system is perfect, but I would rather have the imperfections of the old system, or put multiple scores side-by-side and let people compare.

    Those that know better will read the written reviews, I am just skeptical of how many people know better, particularly the non-member Google searchers.
    MNAle and TongoRad like this.
  15. SLeffler27

    SLeffler27 Grand Pooh-Bah (3,490) Feb 24, 2008 New York
    Pooh-Bah Society Trader

    Here are a few of my thoughts...

    - Reporting the overall ranking is nice, but will be better if the total number of beers is indicated as well.

    - Reporting the ranking by style, with the total of that style would be better yet.

    - Addressing the style bias is a greater concern.
    It might be better to compute "like" as a variable average, plus some constant, and do so by style, rather than all beers. Then weight that calculation as a ratio against all beers. Effectively resulting in, and reporting, two ratings per beer: One strictly by style, the other by all beers.
    The issue of contributing reviewers actually rating/reviewing in comparison to the style will continue to exist, and will still be mitigated by the number of ratings. However, at least the less "liked" styles won't be adversity impacted by taste fashions.
    meefmoff likes this.
  16. Brianito

    Brianito Zealot (643) Mar 23, 2009 Tennessee

    The "Liked this beer" line replacing the one word rating ("Outstanding", etc) is confusing and a step backwards. I very much dislike it.
  17. bobhits

    bobhits Maven (1,462) Oct 31, 2006 Kentucky

    I don't like this one bit.

    A 3.75 is a stellar beer review. Most beers should not and will not get a score like that from me. The "average" beer is after all very enjoyable, but still average.

    This further doesn't tell the story of how many people are "wowed" by a beer and now polarizing beers won't get the benefit of those who really love them vs those who don't.

    The site has and likely will always have a problem with most people scoring rather highly on beers and we can debate if that's right or not. I feel overall one should rate around a 3.0 with the understanding that we don't buy beers we don't like so there should be a slight positive skew. But this creates a whole other mess. Anyway I love the efforts to improve, but I don't think we're there yet.
  18. bobhits

    bobhits Maven (1,462) Oct 31, 2006 Kentucky

    This. I consider a score of at least 3.0 as a statement that the beer was enjoyable but not special or worth chasing down. A 2.0 is a beer that will do if nothing else is available, and below that is where you turn to water or non beer alcoholic products if you still want a drink. To me a 4 is something pretty special and worth looking for, not just a nice to have at a bar. A 3.5 was something really nice to have. After all average commercial beer should be pretty enjoyable for most consumers. If it weren't I doubt beer would be doing all that well.
    hoptheology, BBThunderbolt and Sabtos like this.
  19. SLeffler27

    SLeffler27 Grand Pooh-Bah (3,490) Feb 24, 2008 New York
    Pooh-Bah Society Trader

    There is an inherent problem with wanting others to utilize our personal definitions, or in this case, our personal rating systems, vs. the collective understanding. Whether we like it or not, it seems the typical user of BeerAdvocate has a definition of the rating scale that is more akin to subjective taste, than style guides or objectively agreed standards. Therefore, it seems to me better to accommodate the typical subjectivity, AND have some means by which to cull-out objective opinions.

    The new reporting system may be a step towards both, as it retains the Five Piont scale. The new system provides direction to those who value the collective's trends, while retaining (without improving) the availability of data for others who look for something closer to impersonal analysis.

    While I personally prefer the objective approach, and try to record my observations accordingly, I am under no delusion that my opinion is expert. Maybe the "Bro's" rating can be replaced by a calculation of "expert members." Who qualifies as an expert member would be an interesting conversation.
    Ranbot, BBThunderbolt and drtth like this.
  20. teromous

    teromous Pooh-Bah (2,826) Mar 21, 2010 Virginia
    Pooh-Bah Trader

    I've always gone off the old:

    1: Awful
    2: Poor
    3: Average
    4: Outstanding
    5: World-Class

    If a trait was above average, it was 3.5. I probably won't bother even looking at the new percentages because it just tells me that people rated a beer at or above a certain number. There isn't currently a real metric for what people "like." There is plenty of beer that I find average (3.0) that I still like.

    That tiny average below it is what matters to me.
    BBThunderbolt and Sabtos like this.
  21. SLeffler27

    SLeffler27 Grand Pooh-Bah (3,490) Feb 24, 2008 New York
    Pooh-Bah Society Trader

    Just as a point of clarification, an average beer is not the same as an average rating. As others have noted, the poor beers will be discontinued, due to lack of sales, thus reducing the opportunity for them to be reviewed. I suspect others are like me, in that they select beers they think they will like, thus skewing the sample further .

    BeerAdvocate is a social place, and should find ways to accommodate the social use of the site. This is why I think the subjective, collective is important. I do think there should be progress towards an objective evaluation too.
    drtth likes this.
  22. TongoRad

    TongoRad Grand Pooh-Bah (3,826) Jun 3, 2004 New Jersey
    Pooh-Bah Society Trader

    Then, quite frankly, they're doing it wrong:

    So, unfortunately we're stuck with finding the best way to deal with a gigo situation. This new system only exacerbates the problem, apparently.
    MNAle likes this.
  23. GoodbyeElvin

    GoodbyeElvin Initiate (186) Mar 25, 2010 New Hampshire

    I despise the new "system." I have long-developed habits of evaluating beer based on my experience with the old one. This has the stench of change for change's sake, always a mistake.

    I always found the bros' rating interesting. Now apparently I must search for it in some goddam historical menu.

    Change it back, or I'm on to RateBeer.

  24. bwarner2015

    bwarner2015 Crusader (407) Mar 25, 2016 Connecticut

    I posted my letter grading system a few days ago, and I like the idea also.. Right now I'll be ignoring the % Liked score as I really don't like it.. I will look at the Average, and compare with my letter grading system. Not to critique your letter grading system, but to explain mine better, I looked at my two favorite beers (ddh Congress and ddh Melcher), and they average 4.59 and 4.52, so I put anything above a 4.33 as A+ or World Class.
  25. WesMantooth

    WesMantooth Grand Pooh-Bah (4,748) Jan 8, 2014 Ohio
    Pooh-Bah Trader

    I agree with this. While I won't argue that I, and most users, clearly "liked" a beer that we scored 3.75 or better, I have liked plenty of 3.25-3.5 scored examples. I pretty much consider anything 3 average, and 3.5 or better good. For this reason, you can't assign a threshold for a "liked" beer anymore than you can decide what is "okay", "poor" or "excellent". So like you said, the new threshold was just a lateral move as far as using quantifiers. I will just continue to go by the average scores as I apply them myself like I have since day 1 using this site in the stores long before signing up.
  26. WesMantooth

    WesMantooth Grand Pooh-Bah (4,748) Jan 8, 2014 Ohio
    Pooh-Bah Trader

    I meant to mention that aside from ditching the "liked" denotation, I do think the rest of the changes are improvements. I like the little snap shot of stats. Although it doesn't mean much to me, I would probably include "# _____ out of ____ beers" if you are going to list the ranking. Seeing that a beer ranks 3,197 isn't much of a quantifier. Out of 9,000 beers? Out of 18,000 beers?
    VABA, Oktoberfiesta and Sabtos like this.
  27. MNAle

    MNAle Initiate (0) Sep 6, 2011 Minnesota

    I like :wink: the idea of expert reviewers. I suggest certain reviewers be tagged as "expert" based on:
    • Cicerone certification
    • BJCP certified judge
    • The Alström brothers (because besides being experts, it is their site!)
    • and, BA-certified experts

    For that latter category, members could self-nominate (or others could nominate), and then they would be "voted in" to the expert category by the current expert panel members.

    The expert panel's rating would be displayed at least as prominently as the current "Liked" number.
    #147 MNAle, Sep 25, 2017
    Last edited: Sep 25, 2017
  28. SLeffler27

    SLeffler27 Grand Pooh-Bah (3,490) Feb 24, 2008 New York
    Pooh-Bah Society Trader

    Right or wrong, we should deal with the reality of how the site is used. And regardless of an individuals preference, reality is that the beers people like govern the beers available to us all.
    TongoRad likes this.
  29. Squire

    Squire Initiate (0) Jul 16, 2015 Mississippi

    Just tell me what the rules are and I'll cheerfully comply.
    VABA likes this.
  30. MNAle

    MNAle Initiate (0) Sep 6, 2011 Minnesota

    There are no rules.... just a bunch of guidelines! :grin:
    TongoRad and Squire like this.
  31. Ranbot

    Ranbot Pooh-Bah (2,327) Nov 27, 2006 Pennsylvania

    I would wholeheartedly support this. Not to replace the public ratings/ranks, but to add additional information for consideration, like Rotten Tomatoes users vs critic scores. More knowledgeable members, presumably the experts, would have more incentive to keep returning to this site too. This is in line with the community Beer Advocate has already, which is generally more educated and knowledgeable than the other beer community sites/social media groups. The experts that make this site stand out from the others should be encouraged to stay and contribute. Breweries might also be able to glean more information from expert and public reviews (both have value).
    SLeffler27 likes this.
  32. Squire

    Squire Initiate (0) Jul 16, 2015 Mississippi

    Then I'll draw inside of them.
    VABA and TongoRad like this.
  33. dbrauneis

    dbrauneis Grand High Pooh-Bah (7,470) Dec 8, 2007 North Carolina
    Super Mod Pooh-Bah Society Trader

    You do have to go searching for some "goddam historical menu" to find The Bros reviews - they are linked by "Alströms:" in the Beer Stats section (which is the score they gave the beer) or by clicking on the "Alström Bros" link in the Sort By section of the ratings.
    Lone_Freighter and thebeers like this.
  34. LehighAce06

    LehighAce06 Pooh-Bah (2,104) Jul 31, 2010 Pennsylvania
    Pooh-Bah Trader

    My apologies if I missed this being mentioned, but how about a Peak Rank data point (and perhaps length of time at its peak)?

    Heady has fallen a bit of late, but it's worth noting that it held the #1 spot (and if length-at-peak is included, for SO long). This also would give some much needed context to retired beers that have no current rank, but once were quite high on the list.
  35. LehighAce06

    LehighAce06 Pooh-Bah (2,104) Jul 31, 2010 Pennsylvania
    Pooh-Bah Trader

    You mean the AB InBev shill site?
  36. cryptichead

    cryptichead Grand Pooh-Bah (3,541) Jul 3, 2014 Illinois
    Data Mod Pooh-Bah Society Trader

    After getting used to the new scoring system in the last week, a couple of observations:

    1) EXPERIENCE: I've been primarily looking at and relying on the average score and not giving the new BA score (% liked) too much weight, but still found it somewhat helpful as a general indicator. I've really enjoyed exploring and looking up individual beer's ranks.

    a) I think it would be interesting and useful to show rank movement in the last week/month (i.e., a red or a green arrow showing positions improved or lost). This would be a very helpful indicator of the trend of the beer. Or showing a trend graph of the beer's rating over time (Yelp does this).

    b) I agree with a lot of comments that it would be nice to see a rank # out of total # for that particular beer style. This would provide great perspective and help explain the average and BA score

    c) I really liked the idea that someone mentioned of identifying and being able to sort by "expert" reviews, including the Bros. The Bros Score was the one constant that helped normalize and gauge scores throughout time and across styles.

    Still enjoying the site and improvements! Keep on, keeping on!
    SLeffler27 likes this.
  37. JFresh21

    JFresh21 Savant (1,012) Mar 6, 2012 Illinois

    I'm not a fan of the new scoring. Seems like they are dumbing down the total score. Just give me the rating, % of scores over 3.75 is not valuable.
  38. bwarner2015

    bwarner2015 Crusader (407) Mar 25, 2016 Connecticut

    What is the % liked for the new BA scoring system? lol.
    I will be using the Average Score to determine if a beer is worth buying or seeking out. I would also like to see another layer of scoring that reflects only recent reviews - say within the past 1 year. As the palates of reviewers change, head brewers change, facilities change, ingredients change, better beers are introduced, I would like to see a Total Average, from when a beer was first introduced to present, PLUS a recent Average (past year only). The average ratings for beers with thousands of reviews over many years do not change much, and may not accurately reflect the current ratings for that particular beer. For example purposes only, Heady Topper has over 13,000 reviews going back to 2005, and recent reviews will likely not change its average significantly. I'd like to see what the average reviews are for Heady over the last 1 year only, to see if its ratings have declined or increased compared to its historical average. Not necessarily due to changes in quality, but perhaps due to the variance in certain batches, and also the change in our palates as there are hundreds of new beers to compare with. Does this make sense?
    DonicBoom and Lucular like this.
  39. bobhits

    bobhits Maven (1,462) Oct 31, 2006 Kentucky

    Well I think I like beer that's below average. After all brewers work very hard to make good tasting beer, so I'd expect an average beer to be pretty nice. So for me a 3.0 is not a bad score, it's just that it's average based on my experiences. These scores imply either that a 3 isn't a nice enjoyable beer which seems wrong. Even well below average home brews as long as they aren't infected are still generally acceptable for me. They won't get a 3 though, lol
  40. TongoRad

    TongoRad Grand Pooh-Bah (3,826) Jun 3, 2004 New Jersey
    Pooh-Bah Society Trader

    Oh, definitely, but since this new system has yielded such absurd results it's either a misplaced cutoff number or the system itself that's the issue. I'm not sure which it is at this point.

    And on that last part, it's a double edged sword, at best.
    SLeffler27 likes this.
Thread Status:
Not open for further replies.