Proposed Updates to Top Rated Beer Lists

This.

 

Is anyone here, or does anyone know an actual statistician who might be able to give an informed opinion from a mathematical perspective?

If not, maybe contacting someone in the field about the question would be worth our time.

 

I understand that I am not as integrated into the BA review system as many, so perhaps this more “outside” view is irrelevant, but…from my perspective it seems there are several issues. Aside from a lack of reviews (especially for new beers), it seems like the problem is one of time: How do BA lists stay relevant given the # of breweries, # of new beers, and lack of ratings?

It seems to me like the answer is to focus on timing. Let me explain by highlighting what @Todd mentioned a few posts above:

Why not have a “Top 250”, a “beers of fame” and a “Best New beers (250 or 500)”, the idea being that it would use the date added and first review to start how long it *could* enter AND stay on the list. (I think you need both because some beers, like bourbon county, get announced and added early, but may not be released for 1-6 months). This means that at most a beer would be on the list for no more than a year.

By lowering the threshold of reviews to 5 and creating this new list, you have hyper focus on new, “hip” and “hyped” beers, regardless of where they are or whether they can knock the guardians off their post. You could even throw a factor into the algorithm that takes into account the size of the brewery and whether that brewery is a local, regional, or national distributor. That way, even if Other Half (since someone mentioned them above) releases 40 new IPAs in a year, and even if they all get 25 ratings, they won’t just populate 40 of the spots because # of people to review (ie size of brewery) is taken into account. On that note, you could even put a cap on how many of the brews from a single brewery could be on the list (5? 10?).

Some of the other discussion is still relevant for potentially “fixing” the top 250, but I think the best thing that can be done to solve the issue is to actually create a new list that has beers constantly rotating in/out.

 

Hello! Former property-casualty actuarial analyst here! Ideally, you'd want to apply a credibility factor when aggregating your ratings. More recent experience is presumed to be more credible than older experience and is weighted accordingly. Any experience older than X years isn't credible, other than as a curiosity, and is either marginalized or excluded altogether. Larger numbers are more predictive than smaller numbers. If you happen to share ownership with another company with compatible data, pull that in, too. Todd seems to already be doing some of this but is shy about explicitly saying so. Considering how long it took me to think of the word 'compatible,' and then spell it, I'd probably be wanting a relatively high abv beverage to consider the matter further.

 

I am on team more lists the merrier, so why not have top active, top retired, etc?

 

So essentially a combination of the current Top New and Trending lists?

 

Kind of, but I think the devil is in the details. You look at those lists, and if you pay even a little attention on BA, you’ll recognize a huge number of those beers. Most are from VERY known breweries If you don’t include a measure to offset how available a beer is, size of brewery etc., you’ll continue to see nothing but certain breweries. I mean, that’s cool, I appreciate those lists at times. But I don’t feel like having those solves the issues raised here.

Or, I could always just be missing the point. Missing the trees through the forest, so to peak.

 

I wonder if people like the top rated lists on Untappd. If so, would it make sense to mirror that?

 

No because untappd lists every year a beer is released as a separate entry/vintage

 

Bayesian updating would be an interesting way to help refine older beers staying on lists by taking into account more recent ratings, particularly if they are retired beers. Does BA do anything like this?

 

Is there any potential for this? Seems like a good idea given that first ratings are often exaggerated, and as time goes on ratings usually drop (some) due to age. Seems like it would make some sense, even if you don’t put it into effect until a beer has reached a certain age, for example.

 

ie Bought the beer off the shelf of my local beer store

 

A store I was just at would qualify as well. I think 90% of the beer there was really functioning as insulation for the walls rather than purchasable product. It’s crazy.

 

I read all of the above, even though I now realize I failed to give a "like" to some good comments. I'm good with the proposed changes as long as there is an indication of the number of reviews that went into the rankings. Clearly, a beer that has only 5-6 ratings/reviews is subject to having an overall score skewed by inclusion of (for want of a better term) an uneducated or unrealistic reviewer. When the minimum number or ratings is 10 that is somewhat less likely to happen. I have no answer to the dilemma of how to exclude individual ratings made by morons, fools, or those who don't give a hoot what the intended rating system is supposed to be based upon, like the rating from a person who assigns a "2" to a sour when they already know they hate all sours. How about for starters, we require 5 reviews, rather than 5 ratings. A review can at least provide insight to the validity of a score, although I recognize that reading all the reviews is unrealistic.
I'm a little conflicted about a change to now include scores that are more than +/- "x" standard deviations from the norm. I somewhat support excluding outliers, although the threshold for what constitutes a real outlier could be reviewed.

 

That alone makes it sound like a really fun exercise if there was someone like a BA volunteer who could grind out the numbers for free. I think it would be as interesting as heck to know how average scores from reviewers compare to those of tickers, at least for a subset of beers, especially those which have high standard deviations in scores. However, I fully understand the need to prioritize things when resources are limited. I like the opportunity to provide input on these ideas, though