New BA Score, Beer Ranking & More Updates

Another local example, this time from the same brewery.

3.98 rated IPA vs. a 3.77 rated IPA
86% liked vs. 63%. For a beer that is rated only 5% better, there appears to be a 4 times multiplier in enjoyment. At least thats the perception that they are now giving by BOLDING the % liked.

Each 0.10 difference is amounting to some 10% change in % liked, for those beers within that 3.75 threshold. To me that feels like a flawed threshold. It's just said a totally respectable beer like SA Latitude 48 now sits at 42.5% liked (r score of 3.56)

Another example. Ranger IPA vs. Lagunitas IPA. Ranger is a 3.77 rated beer and 65% liked. Lagunitas IPA is a 3.89 rated beer and 76% liked. It's not like this for every beer. But every 0.01 change in the R, we're seeing the % like increased by 1% point.

There's too much variance around 3.75. So much so that when you look at the top 95%- 99.5% liked beers, you really can't distinguish between them. We're seeing some beers in the 4.35 rated range coming in nearly the same % liked as a 4.60 rated product. If we compared the middle to the top end, we should see a 25% difference in % liked. A 97% enjoyed 4.35 beer should compare to a 4.60 122% enjoyed beer. Obviously that's above the 100% max. But it just shows to me anyway that something isn't right.

When it comes down to it, we're seeing 3.52 rated beers coming in at 42% liked, then 0.20 rated higher beers coming in at 63% liked, and then 0.20 beers above that, 85% range. At a certain point, your 4.60 perfect beers should be ABOVE PERFECT. That's what I'm getting out of it.

I like the muddling of the numbers on the top of the spectrum where 0.25 difference in scores only amounts to 2% difference in enjoyment. That part I can understand. But the 3.75 marker is just brutalizing so many beers that the % liked becomes a joke

Ill stop beating the dead horse now.

 

Arcadia Brewing Company Bourbon Barrel Shipwreck Porter. Thanks for the assistance.

 

and to take your thought one step further - will this encourage more users to rate to style?

makes me wonder....

but a guy can hope....

 

Nice! Thanks for doing a lot of legwork on this.

 

I doubt it. I saw someone rating some beer of a style that was supposed to have little to none hop profile and he dinged the hell out of it because of this. Not sure that will ever work it's way out. I have been going back tru the beers I had when I first started drinking good beer and trying to correct the dumbass things that I did. It is what it is, I can deal with the %, because I am always going to look at the number rating and the reviews, but google grabbing that % and throwing it up on the search page is killer.

Off topic, but regarding Google. If you have Android Auto you can ask Google about a beer and she will read you the BA Notes section. Cracked me up. Kept me busy from work all the way home the other day.

 

Yup, guess that was a pipedream, oh phooeeey.

Wait a minute, was this backlash because the Bro's changed the way the scores are done? Sounds pretty immature on that person's part. There's the "report abuse" button on the bottom left of people's reviews, ya know.

But still my hat goes off to you for going back through your beers, that's a respectful thing yet possibly a daunting task to do something like that.

And that's pretty cool thing that Google does! I have an Android, I'm gonna have to check this out. Thanks man!



-----------------------------------------------------------------------------------------------------------------------

sidenote: and I'm not asking to be lynched on BA by anyone as I may be one of the few that doesn't think negatively or positively about the scoring changes yet. There's pros and cons and well, I'm just going to go with the flow for the most part to continue to see these things to fruition. There's always growing pains in changes and I'm not disrespecting anyone who has been beer reviewing longer than me that may think differently.

 

No, while back I saw that, just a random review I was reading while deciding to purchase the beer. I think the thinking is, which can be problematic, that you simply rate a beer how you like it, not how well the beer is brewed to style (tons of threads debating this). That was the review that clicked that I needed to be more careful with how I review a beer. If I tried something and didn't like it, I needed to figure out if it was me or if that is the way the style was intended to be. As a matter of fact in looking at my rating for Pilsner Urquell I slid that puppy in there right above the line at 3.77. I had that about 1 year into my current beer craze. Didn't appreciate a pilsner the way I do now. But also not sure what shape that beer was in when I had it. Having drank a ton of it on tap when I was in Poland, I should have updated my review, but didn't (there was 1 place there that had only it on tap and poured it like 3 different ways, various levels of head).

 

Completely agree. How about another score that shows where a beer ranks within its style? A couple ways to do it:
El Sully (a good example, it has a low 'like' rating but ranks well among AALs)
Avg: 3.46/5 (style avg: 3.00)
-or-
90th percentile in American Adjunct Lagers

 

Just to reinforce what @donspublic has said, even though I've not seen the particular review he mentiond, it's not likely to be backlash. I've been seeing such reviews for "lo these many years" now. I've decided it's an endemic problem for a site like this where there's no "entrance exam" or "training course" to qualify anyone as being a rater/reviewer.

Many folks aren't looking to learn and so don't bother with the Beer 101 information and just start using the numbers, etc. So reviewing while "keeping style in mind" is something a lot of folks do try to do, others....

This is one reason I only pay attention to reviewers I trust when making up a shopping list of stuff I think I might like to try. (The other being that I'm cheap and don't want to be spending money on beer I won't enjoy. )

 

I may have been confused initially by what Donspublic was saying but I get it now.

And yes, my guess the ratio of cicerones to regular beer drinkers is probably so far imbalanced towards the regular beer drinkers that you have a valid point in understanding that so many people most likely rate to personal preference in the beginning (we all have to start somewhere, right?). But there's probably also a fair amount of people that continue on to rate to personal preference too.

I'd like to think that as people grow in their understanding of beer styles that many would want to come back to beers that they didn't understand at one time to try again to see if they like it the second time. Makes me wanna go back and see my own reviews to see what I can do better.

 

If I'm having a beer I first reviewed "back in the day," I'll often take a look through the review to see if I want to update it or add to it, but otherwise ignore that urge and satisfy myself by concentrating on continuing to develop my understanding so that I can convince myself that I've been improving.

 

Same same

 

I agree with all of this. No system is perfect, but I would rather have the imperfections of the old system, or put multiple scores side-by-side and let people compare.

Those that know better will read the written reviews, I am just skeptical of how many people know better, particularly the non-member Google searchers.

 

Here are a few of my thoughts...

- Reporting the overall ranking is nice, but will be better if the total number of beers is indicated as well.

- Reporting the ranking by style, with the total of that style would be better yet.

- Addressing the style bias is a greater concern.
It might be better to compute "like" as a variable average, plus some constant, and do so by style, rather than all beers. Then weight that calculation as a ratio against all beers. Effectively resulting in, and reporting, two ratings per beer: One strictly by style, the other by all beers.
The issue of contributing reviewers actually rating/reviewing in comparison to the style will continue to exist, and will still be mitigated by the number of ratings. However, at least the less "liked" styles won't be adversity impacted by taste fashions.

 

The "Liked this beer" line replacing the one word rating ("Outstanding", etc) is confusing and a step backwards. I very much dislike it.

 

I don't like this one bit.

A 3.75 is a stellar beer review. Most beers should not and will not get a score like that from me. The "average" beer is after all very enjoyable, but still average.

This further doesn't tell the story of how many people are "wowed" by a beer and now polarizing beers won't get the benefit of those who really love them vs those who don't.

The site has and likely will always have a problem with most people scoring rather highly on beers and we can debate if that's right or not. I feel overall one should rate around a 3.0 with the understanding that we don't buy beers we don't like so there should be a slight positive skew. But this creates a whole other mess. Anyway I love the efforts to improve, but I don't think we're there yet.

 

This. I consider a score of at least 3.0 as a statement that the beer was enjoyable but not special or worth chasing down. A 2.0 is a beer that will do if nothing else is available, and below that is where you turn to water or non beer alcoholic products if you still want a drink. To me a 4 is something pretty special and worth looking for, not just a nice to have at a bar. A 3.5 was something really nice to have. After all average commercial beer should be pretty enjoyable for most consumers. If it weren't I doubt beer would be doing all that well.

 

There is an inherent problem with wanting others to utilize our personal definitions, or in this case, our personal rating systems, vs. the collective understanding. Whether we like it or not, it seems the typical user of BeerAdvocate has a definition of the rating scale that is more akin to subjective taste, than style guides or objectively agreed standards. Therefore, it seems to me better to accommodate the typical subjectivity, AND have some means by which to cull-out objective opinions.

The new reporting system may be a step towards both, as it retains the Five Piont scale. The new system provides direction to those who value the collective's trends, while retaining (without improving) the availability of data for others who look for something closer to impersonal analysis.

While I personally prefer the objective approach, and try to record my observations accordingly, I am under no delusion that my opinion is expert. Maybe the "Bro's" rating can be replaced by a calculation of "expert members." Who qualifies as an expert member would be an interesting conversation.

 

I've always gone off the old:

1: Awful
2: Poor
3: Average
4: Outstanding
5: World-Class

If a trait was above average, it was 3.5. I probably won't bother even looking at the new percentages because it just tells me that people rated a beer at or above a certain number. There isn't currently a real metric for what people "like." There is plenty of beer that I find average (3.0) that I still like.

That tiny average below it is what matters to me.

 

Just as a point of clarification, an average beer is not the same as an average rating. As others have noted, the poor beers will be discontinued, due to lack of sales, thus reducing the opportunity for them to be reviewed. I suspect others are like me, in that they select beers they think they will like, thus skewing the sample further .

BeerAdvocate is a social place, and should find ways to accommodate the social use of the site. This is why I think the subjective, collective is important. I do think there should be progress towards an objective evaluation too.

 

Then, quite frankly, they're doing it wrong:
https://www.beeradvocate.com/community/threads/how-to-review-a-beer.241156/

So, unfortunately we're stuck with finding the best way to deal with a gigo situation. This new system only exacerbates the problem, apparently.

 

I despise the new "system." I have long-developed habits of evaluating beer based on my experience with the old one. This has the stench of change for change's sake, always a mistake.

I always found the bros' rating interesting. Now apparently I must search for it in some goddam historical menu.

Change it back, or I'm on to RateBeer.

Ridiculous.

 

I posted my letter grading system a few days ago, and I like the idea also.. Right now I'll be ignoring the % Liked score as I really don't like it.. I will look at the Average, and compare with my letter grading system. Not to critique your letter grading system, but to explain mine better, I looked at my two favorite beers (ddh Congress and ddh Melcher), and they average 4.59 and 4.52, so I put anything above a 4.33 as A+ or World Class.

 

I agree with this. While I won't argue that I, and most users, clearly "liked" a beer that we scored 3.75 or better, I have liked plenty of 3.25-3.5 scored examples. I pretty much consider anything 3 average, and 3.5 or better good. For this reason, you can't assign a threshold for a "liked" beer anymore than you can decide what is "okay", "poor" or "excellent". So like you said, the new threshold was just a lateral move as far as using quantifiers. I will just continue to go by the average scores as I apply them myself like I have since day 1 using this site in the stores long before signing up.

 

I meant to mention that aside from ditching the "liked" denotation, I do think the rest of the changes are improvements. I like the little snap shot of stats. Although it doesn't mean much to me, I would probably include "# _____ out of ____ beers" if you are going to list the ranking. Seeing that a beer ranks 3,197 isn't much of a quantifier. Out of 9,000 beers? Out of 18,000 beers?

 

I like the idea of expert reviewers. I suggest certain reviewers be tagged as "expert" based on:

• Cicerone certification
• BJCP certified judge
• The Alström brothers (because besides being experts, it is their site!)
• and, BA-certified experts

For that latter category, members could self-nominate (or others could nominate), and then they would be "voted in" to the expert category by the current expert panel members.

The expert panel's rating would be displayed at least as prominently as the current "Liked" number.

 

Right or wrong, we should deal with the reality of how the site is used. And regardless of an individuals preference, reality is that the beers people like govern the beers available to us all.

 

Just tell me what the rules are and I'll cheerfully comply.

 

There are no rules.... just a bunch of guidelines!

 

I would wholeheartedly support this. Not to replace the public ratings/ranks, but to add additional information for consideration, like Rotten Tomatoes users vs critic scores. More knowledgeable members, presumably the experts, would have more incentive to keep returning to this site too. This is in line with the community Beer Advocate has already, which is generally more educated and knowledgeable than the other beer community sites/social media groups. The experts that make this site stand out from the others should be encouraged to stay and contribute. Breweries might also be able to glean more information from expert and public reviews (both have value).

 

Then I'll draw inside of them.

 

My apologies if I missed this being mentioned, but how about a Peak Rank data point (and perhaps length of time at its peak)?

Heady has fallen a bit of late, but it's worth noting that it held the #1 spot (and if length-at-peak is included, for SO long). This also would give some much needed context to retired beers that have no current rank, but once were quite high on the list.

 

You mean the AB InBev shill site?

 

I'm not a fan of the new scoring. Seems like they are dumbing down the total score. Just give me the rating, % of scores over 3.75 is not valuable.

 

What is the % liked for the new BA scoring system? lol.
I will be using the Average Score to determine if a beer is worth buying or seeking out. I would also like to see another layer of scoring that reflects only recent reviews - say within the past 1 year. As the palates of reviewers change, head brewers change, facilities change, ingredients change, better beers are introduced, I would like to see a Total Average, from when a beer was first introduced to present, PLUS a recent Average (past year only). The average ratings for beers with thousands of reviews over many years do not change much, and may not accurately reflect the current ratings for that particular beer. For example purposes only, Heady Topper has over 13,000 reviews going back to 2005, and recent reviews will likely not change its average significantly. I'd like to see what the average reviews are for Heady over the last 1 year only, to see if its ratings have declined or increased compared to its historical average. Not necessarily due to changes in quality, but perhaps due to the variance in certain batches, and also the change in our palates as there are hundreds of new beers to compare with. Does this make sense?

 

Well I think I like beer that's below average. After all brewers work very hard to make good tasting beer, so I'd expect an average beer to be pretty nice. So for me a 3.0 is not a bad score, it's just that it's average based on my experiences. These scores imply either that a 3 isn't a nice enjoyable beer which seems wrong. Even well below average home brews as long as they aren't infected are still generally acceptable for me. They won't get a 3 though, lol

 

Oh, definitely, but since this new system has yielded such absurd results it's either a misplaced cutoff number or the system itself that's the issue. I'm not sure which it is at this point.

And on that last part, it's a double edged sword, at best.