Proposed Updates to Top Rated Beer Lists

I feel lonely and irrelevant when my favorites actually disappear from the shelves and taps.

 

I feel lonely and irrelevant. I drink beer.

 

Maybe you could be the fifth Beatle!?!

 

It's not really a good comparison though. If you look at the top 10 NEIPAs on BA, 8 of them are from Tree House, and the average number of ratings for those beers is 1.5 thousand per beer. If any of those beers had only 5 ratings instead, I'd say that the number of reviews wasn't large enough to have those beers qualify for the list despite whatever sense of "general acknowledgment" I thought existed for Tree House. BA lists are about BA stats. If people feel that only 5 ratings are indicative enough of general acknowledgment, then so be it.

As you pointed out, if the issue in your country is consumer interest in BeerAdvocate (just as it is for @misteil in Ireland), then the appropriate solution is to generate more consumer interest (albeit clearly easier said than done) rather than lowering the ratings bar across the board regardless of region. If the consumer interest is so small that even getting 10 ratings for a beer is an issue, then why does it even matter if the BA list changes for that region due to the proposals?

 

Glad to see the raw average being used. Also glad that the minimum threshold has been dropped to 5. Makes sense to me given the sheer number of beers being potentially added.

 

Nah man you’ve this wrong, at least in the Irish context. Example, the best brewery in Ireland is Whiplash, there’s no argument to be had they’re just legends. One of their beers has more than 10 reviews, and thus is the only one in the top 100 for Ireland. Change it to 5, 7 of their beers end up in the top 100 (out of the active ones, that is). Surely you would agree that if 7% of the Irish top 100 was Whiplash, this would increase consumer interest? Not significantly I am sure, but it would likely be a help to some degree. The difference of 5 and 10 may not matter in some regions, but it just would in Ireland, the top 100 right now is filled with crap, and this change would make a significant difference.

 

@Todd - In the companion thread to this one, I was mentioning that "inactive" and "retired" as two different categories felt cumbersome or confusing... so my knee-jerk reaction is to agree with consolidating them (but perhaps let the dust settle first). As it is, the current criterion for them together doesn't feel totally cohesive (in that they each have their own system and reason for being rather than being 2 categories in one logic system). But your consolidation idea above has an issue. An inactive beer would automatically change to active with a new rating... so if you do away with "retired" and someone rates an aged beer that hasn't been brewed for a decade, it would become "active" on a brewer's list.

But perhaps the larger beer categorization system on BA is too stuck in the past and doesn't reflect how many beers are released these days. "Retired," "one-off," and "rotating" (to bring in the quote above from @dbrauneis and another variable) increasingly seem like solutions to a previous landscape. For a majority of beers, how would a consumer even know this info (or even brewers for that matter)? Is it increasingly irrelevant or inaccurate? I don't know.

 

True enough. As I mentioned, I don’t know if any of these changes will help make the lists more indicative of the beer reality here or elsewhere. I’d like to see if they do.

My mentioning of Tree House was more to illustrate that breweries generally regarded as the best makers of a popular style do find themselves on some of the US and Global style lists. The best breweries here making the most popular beer style right now are almost wholly absent from the Canadian list. (At last look there are a couple older Superflux beers on it). The Canadian list is Canadian Beers of Fame list by default. Interesting to be sure, but not comparable to some of the US lists in terms of contemporary relevance.

As you say, getting more Canadian users to rate and review here is the real answer to the issue. I’ve been encouraging people I know to do this for years. Sadly, I know far more Canadian users/former users who have left for Untappd and Facebook groups than I know new users who join and stick around here. Most new beers I rate have 0-3 other ratings/reviews from the same pool of 4-5 users. There were far more active Canadian users 7-8 years ago than there are today. So, the list as it stands will likely always reflect beer tastes from those years more than anything else.

In the past, I’ve found the US lists ( in combination with the regional forums) helpful when travelling. I can get a sense of some of the things to look out for. If a lower rating threshold might produce a list that more accurately reflects what even the limited number of Canadian users see as our best beers, then I think it might matter greatly.

 

If I'm wrong, then why not lower the threshold to 1 rating to get even more of the "crap" off the list and more of the "legends" on it?

 

I figured as much on this - I'd say when in doubt select it as one-off, because if it does become an annual release or come back into the rotation, it's more likely to get identified as not really being a one-off than the reverse.

I like the new setup to have "inactive" and "retired". The former is set up as a function of activity on this site, but the latter is the override switch. Seems pretty functional to me - I'd hate to have it only be the "inactive" switch. For example, if someone rates Don Quijote tomorrow - it shouldn't jump back into the lists!

 

It think I am getting your meta-message here but please correct me otherwise.

It seems that the number of 'hype' breweries that are constantly created new (or SBDL - Same Beer Different Label) beers on a regular (e.g., weekly) basis is challenging situation. I personally have never had an Other Half beer but it seems like they come out with new 'one-off' beers on a regular/weekly basis. And if all you need is 5 ratings (likely all 5's for the fans of Other Half?) then there seems like the large number of brands from Other Half (and other similar breweries) will be constantly added to the 'best of' list? Maybe that is OK but...

It also seems that due to the above dynamics and the difference in BA participation between US and non-US users presents another set of challenges.

Cheers!


P.S. I checked to see how many Other Half beers are on the New York top rated list and by my reckoning it is 43 of the 100.

https://www.beeradvocate.com/beer/top-rated/us/ny/

Maybe they will be over 50 with a change here?

 

I wasn't thinking of that or thinking that specifically.

 

I agree with you that conceptually at 5 ratings; it's just too small to be enough of a sample size to start giving that beer a score. Frankly, 10 ratings seems low to me.

I don't know how much it will change the more built up lists though. The formula for the lists factors in how few/many ratings a beer has. Look at Illinois' top 100. BCS Blanton's has 20 ratings at a 4.58, and it's not on the list (I'm guessing it will make it soon with a few more ratings). The 100th beer is a 4.33/56 ratings, and there are some with 4.2-ish and 100+ that are obviously ahead of it based on BA's formula for these lists.

 

OK, just my thoughts then here.

Cheers!

 

Thanks for pointing that out. I was heretofore unaware that there is some other sort of minimum number needed to be placed on a 'best of' list.

Cheers!

 

So many beers now, especially Murkbombs/Murkbrewers, are one-and-done, unless they sell really well, that I always enter a beer as Rotating/seasonal, unless I know for sure that's gonna be either full-time or a one-off.

 

It's not really a minimum to make the list - it's just that the formula weights how many ratings a beer has or hasn't. A beer with 11 reviews could theoretically make the New York list, but even if it had 11x 5.0 ratings, it wouldn't calculate enough to crack the list.

For West Virginia, it would since there aren't 100 beers on that list.

Moving the threshold to 5 wouldn't change a single slot on the New York list for the reasons outlined above.

 

Can you please expound on this?

You state "A beer with 11 reviews could theoretically make the New York list..."

But then you state "...even if it had 11x 5.0 ratings, it wouldn't calculate enough to crack the list."

So, how theoretically could a beer with 11 reviews make the New York list? Is it a function of beer style (or other factor)?

Cheers!

 

I tend to do the same thing. I think I entered one the time I was there last year as a one-off. This time they had beer merch with the logo of that beer. I guess it's a regular. Hard to tell with small, local breweries.

 

Look at the colorado top 100. It's loaded with retired beers that are irrelevant to Colorado beer culture as it sits now.

 

well it’s a kind of bandwidth issue isn’t it. 10 gives a suboptimal result because there simply isn’t a great enough population of people drinking and reviewing the beer, but 1 is suboptimal because one person reviewing the beer is statistically inadequate. 5 seems like the right trade off between statistical significance and efficiency of result of the top 100 list, in my opinion.

 

Yeah, one of my locals has beer that is basically their flagship, but they've never brewed it the same way twice. Different beers, same name.

 

Wow I just now got to look at the Top 250….

Adding back in the raw average instead of truncated mean has really shaken the order of things up. I think this change has shaken up the list more than any of the PROPOSED edits will do. Not saying it’s good or bad, just an observation

 

At least with the inactive portion available, it would be easier to go thru and flag these, as mentioned in the other thread.

Just need some man hour(s) to get it done.

I know I've used the inactive list to report at least a dozen retired beers. I plan on getting around to more with some of my upcoming time off.

 

Once a beer gets to 10 ratings, it's "eligible". But BA has a formula to determine the list, and 11x 5.0 isn't still going to score lower in the formula than the 100th ranked beer on that particular list.

 

Thanks for that additional input.

Needless to say but quite a confusing (for me ) situation here.

Cheers!

 

I often don't know myself when I add new beers either, but I don't really find this as a big deal to me. It all works itself out so I wouldn't beat yourself up over it. This category I find is just for some minor informational purposes I feel in my opinion.

If a beer is a one-off or seasonal, whether that information is correct or not does really mean much to me personally.

Long story short, If I know it, I will chose the right info, if I don't know if it's seasonal, one off or whatever I just guess and don't concern myself with it.

 

Hey we all know who that is... that's not a mystery...

 

Yes siree Bob.

Cheers!

 

Obviously opposed… biased because I’m so damn close to having them all ticked.

 

Imma ask a silly ass question. Isn't this kind of like ignoring the statistics of Babe Ruth and Ty Cobb because they're retired? Only Bryce Harper and Jose Altuve matter because they're active? Is the rookie of the year the most relevant because he's new? Just throwing it out there.

 

The analogy here is that the Top 250 list are All-Stars, while the Hall of Fame is the "Beers of Fame" list.

 

Not really….The beers of fame list does not include retired beers. Once a beer is retired it no longer receives a ranking. Beers of fame simply means the beers were added 15 or more years ago….but still active

 

Just my two cents:

• Do not consider active beers only. If an inactive beer generated a response effusive enough that it isn't unseated from its top rated position after two bloody years (all while the entire user base is rating other beers and the forum's hype trains for new releases keep on choo-chooing), maybe it's fair to say that that beer earned its ranking?

• Do not lower the minimum threshold. If you don't have enough user engagement for a top tier beer to get a scant 10 ratings, then the problem is a user engagement issue, not an algorithmic issue with top rated beers.

I don't think the threat to the top rated beers lists with respect to users discovering new beers is the Top 250 list tickers. My concern is small groups of colluding fanboys will be able to hype a one-off single keg release from their favourite hometown nanobrewery by putting up perfect scores and landing the beer on a list.

If anything, raise the minimum threshold. These lists will only be treated as the de facto best beers lists worldwide (be it by BA's membership, by retailers posting the list(s) on their walls, or by the casual consumer/non BA member who prints off a list one random year to naively bring into their local bottle shop) so long as they actually do reflect the best beers.

My concern with a five rating threshold is that all it would take for a beer to make a list would be the equivalent of a table full of neckbeards using Untapped at the local taproom. Not exactly the authoritative "benchmark for beer reviews" BeerAdvocate is intended to be.

 

@Todd correct me if I'm wrong, but I thought the top beer lists used a weighted algorithm to formalize BA score, with one of those factors being number of reviews and ratings.

This could silence a lot of the doubt, i.e. five users couldn't recreate a top list.

FWIW, I tried to dig up the answer where I thought I saw it, but turns out my memory and internet archives aren't as deep as @jesskidden and others.

 

I would personally like to see the minimum number of reviews dropped, as I have taken the time to review many brews that will likely/never get to ten, but where five is doable....I know I'd like to see a score with more of these types. I'd also like to see inactive beers stay in the rankings on top beer lists.....some people may be aging brews that are no longer available, and others may discover a said beer this way, by acquiring one through trade or by chance finding one on a shelf somewhere....I've found beers no longer brewed before myself.....If its good enough to have made a top beers list, it should stay there.....what about a top 250 list specifically for inactive/retired beers?

 

I'm definitely in support of that, regardless of how it impacts top lists and other site features. When I add a new beer to the DB, I rarely have any clue if the brewery views it as a one-off or plans to make it again, either on a sporadic basis or seasonally. Hell, even the brewery may not know until they see how it sells.