Thoughts on the latest updates? Top 250, etc.

My solution, make the first person to review/rate the beer, or maybe the first few people write a review. This will require a few more reviews but still get tons of statistical data points.

To make a good determination of a beer, I want an average based on >50 scores, preferable 100. Then I will combine that with 2 or 3 reviews that I'll quickly skim through. With the change, the rating will likely only be based on 1-20 scores (or none, as I see a lot of beers have been blanked out) and then I have to sift threw a bunch of junk reviews to find a good one. Overall, it just made this site NOT my go-to for figuring out good beers I want to try.

 

I doubt that, they can easily add 150 characters of something either while hammered, or the next morning. My bigger concern is now a lot of people who like to quick add a rating while at a brewery won't bother, or even if they do, the data;s not there for the next person. They will go to the brewery page and see half the beers have no ratings, the other half only based on a handful of scores so close the page and head over to untappd to get a better/quick idea of which beer is good.

For example, pick your favorite local brewery (mine is Penrose) and see how many beers now have no average score displayed on the brewery page because they were based on only "hads" and the rest are based on 1-10 total scores, not something you can quickly discern what you should order without reading each beer.

 

This isn't fun anymore! We've degraded ourselves to arguing over statistical analysis. Beer is supposed to be fun. Hey if someone finds Natty Light a 5 star beer because, they love it, who cares! Most people want to enjoy their beer, not review it! I only look at a small handful of reviewers on this site for some helpful insight. I don't like it but, in the end, it's "their" web site, they can do what they want with it!

 

From a

From a physics point of view I guess the lower "quality" assumed from quick ratings is like the systematic error from the detectors we have, which are not perfect and introduce bias and error that we try to take into account as best as we can. But the biggest problem we have is the lack of statistical data. We always need more data and spend millions to get it, therefore I might be a little biased, but I think more data is always better than less. I even said this in the beginning of the thread, like you mentioned, getting both qualitative and quantitative results is the best. So some sort of weighting applied to the quick ratings and including them in the total average would be the best solution.

 

Well, adding a 150 characters certainly takes a lot more effort while than giving a quick score.

I think that this will lead to better quality scores in time. It just going to take a long time for some really good local beers with limited distribution to get the BOS respect they deserve.

 

The issue as I observe it in the data is that it doesn't inflate all scores evenly - hence 90ish beers changed on the list. There is a distinct skew in the beers that get the high ratings from non-reviewers.

If it just pushed all scores up evenly then it would be good old grade inflation, but the fact it isn't even across all beers is (at least to me) why it is an issue.

 

Yeah, this is pretty terrible. Looks like the majority of people don't like this change but i doubt BA will listen to the majority of the people. I just went to go rate Trillium Summer Street. Only 6 reviews determine the overall rating now instead of the 130 hads. Those 120ish "hads" were valuable data and now they are tossed out completely. I absolutely agree with the person that has been stating that the more ratings the better because it is more statistically significant. BA may be going back to its roots but times change and this is a terrible decision. I imagine many people do ratings on their smartphones these days. I am usually drinking beers with frends and i am not going to take the time to type out reviews on my phone.

 

I agree, this is definitely not fun, I want to be able to drink good beer and easily find such good beer on BA. Before this change, I could easily search for a beer from a local nano brewery (like Moustache Brewing) and get an idea of the beer even if only a small handful of people rated the beer, now there is no weighted average or score for most beers and I have to scroll through the ratings and calculate an average in my head. Overall, I'm sure this is not a very useful and productive change for the countless new breweries that have opened up in the recent years. Only old traditional and popular beers will retain ratings and scores under this new system.

 

Our bickering about top 250 lists, while important to us, aren't that big of a deal in the grand schme. This issue of not being able to identify good beers from the brewery page is a BIG deal that the site should care about. This site is no longer useful for identifying good beers within a brewery quickly, which means people will go elsewhere to get that info (aka untappd, which I personally can't stand). Making the site less useful to the majority of users is a negative to the site, regardless of how that is happening. There are other easier ways to force more reviews without making the overall averages for most beers, espcially the tiny breweries, worthless.

 

Terrible terrible terrible change. No need to repeat what everyone else is saying, but this top 250 is useless. Admitedly there were some outliers on the high and low end before, but those averaged out and the top 250 used to be a fairly accurate representation of the pecking order of quality beers.

 

Over time that will improve - right now you are at day zero. Over time as fewer people tick, and more people are actually putting in some info and giving a review I bet you will see much more going into each rating.

Admittedly this will take a little time, but I personally think the ratings will be better for it.

 

This statement is extremely presumptuous, assuming that people will now feel forced to enter full 150+ character reviews in order to be counted. It also ignores the fact that this change completely ignores a large amount of completely useful quick rating data for no good reason besides a biased view that they are unworthy simple because they didn't take the time to enter 150+ characters of supplementary text. As @UWBadgerFan4Life pointed out above, mobile usage is on the rise and will soon dominate, and with this comes more use of quick ratings. Trying to force the site to stay in the ancient past is pointless, you should embrace the future and adapt. As much as I hate Untapped, they have succeeded in capturing the masses because of their easy mobile app. Do you really want to keep BA to be anchored to the past so that it will soon become irrelevant? Wouldn't it be better to hold on to some elements that make it superior to Untapped, while at the same time appealing to the larger audience and keep it relevant?

 

It is pretty weird to see so many hads with so few reviews and only have the reviews count towards the score. Hopefully people will start to enter the minimum reviews going forward and help build the website with quality content instead of just a number.

 

I dislike the new system. I do reviews on about 20% of beers I drink. I do them when I believe they will be relevant to a discussion of the beer. If a beer is average, I give it an average rating and move on. Why should I do a review of an average beer? For me, do a review to explain a great or subpar rating, or to describe particular aspects of the beer that can't be explained in the number system. As someone said earlier, the new system takes the fun out of the ratings system. If I go to a bar and get a 5 beer sample, am I going to do a review on each one? NO! I'm there to enjoy myself and some beers, not sit on my phone typing away. I good friend of mine on BA has rated over 2600 beers and I check his ratings to see what he likes. He reviews like me, about 20% of the time. Considering that he's had so many beers, I greatly appreciate his insite. Now over 2000 of his ratings aren't counted in the final score? I think thats a waste of a valuable BA members insite. Its beer, not a case study in medicine! Already talked to 1 person who's thinking of going to another site to rate beers. Hope this changes and doesn't ruin BA.

 

All i've learned from this thread is that people put entirely too much faith in the top beer lists. They're just a list. Not to mention that the list is (as before) ridiculously US-biased, but that's for another thread...

 

Lots of back and forth in here about the benefit or lackthereof from this change in how scores are calculated. I'm sure I'll continue to develop my opinion as I experience the change through my use of the site, but my initial reaction is optimistic. This is primarily because I have always taken the view that lists like these--that attempt to synthesize subjective opinions into a list that gives the appearance of objectivity--are inherently fluid and context-dependent. They are dependent on the formula on which they are based as well as the behavior habits of the people whose opinions comprise them.

Suggesting that the list is better or worse now because it is a truer or less true representation of the "actual" top 250 beers demonstrates, to me, a fundamental misunderstanding of the nature of this type of list. Ultimately, there is no such thing as a top 250 beers--not only will the list depend on the formula used to calculate it, but it will depend on how the people who generate the scores use the formula. Yes, as a general rule, more data = better data, but this rule is not absolute. A smaller data set of higher quality data can be more informative than a larger set of lower quality data. And that also depends on what is considered "quality"

Of course, the question is whether or not the data provided by reviewers is higher quality than the data provided by raters. Personally, I think there is at least some evidence that suggests it might be:

1) Reviewers definitely spend more time and effort describing a beer, which means they likely put more time and effort into evaluating it in the first place. Qualitative evaluation--which is what we are doing, whether we are reviewing or rating a beer--is better with more focus and attention given to the evaluation.

2) Building on the prior point, evaluation of a beer based on having 2-4 oz of it on a day/night where the person consuming it had 2-4 oz pours of 20-something *other* beers in addition to the beer in question isn't likely to be very true to the nature of the beer being evaluated. I personally don't rate/review beers on this site that I've had at a beer festival for this very reason, and this change in how scores are calculated seems to align with the philosophy that the rating for the beers one has at a festival not being terribly valuable.

All that said, I can certainly understand the issues some have expressed here. As has been demonstrated, the change has had a definite effect on certain breweries--many of which happen to be the ones that light up the trading forums on a regular basis. If you are a BA who used/uses the Top 250 to aid in your trading hobby, then at the very least, you're going to have to find a new way to use it. However, it does beg sort of a chicken or the egg question--did breweries like Hill Farmstead, de Garde, Side Project, Trillium, etc. become regulars on the trading forums as a result of their shooting up the Top 250 list, or did they shoot up the Top 250 list because of their popularity on the trading forums. I personally suspect it's the latter, but I'm speculating.

I will say this, one of the reasons I like the change is because I've gotten lazy about my reviews over the past few months, and this change has really rekindled my desire to write reviews, because it really gives the sense that reviews are valuable, and at the end of the day, that seems like what the change was attempting to promote: the value of reviews. And to tie this back to something I said in my opening paragraph--I don't think there's anything wrong with using the methodology for calculating beer scores to promote user behavior, because there is no "true" Top 250 list. The only thing that remains to be seen at this point is whether or not the strategy works.

I these are both perfectly reasonable approaches to take. But I'd also argue that the ratings that come from people just giving an on-the-spot evaluation of a couple ounces of a beer they've tasted amongst a ton of other beers aren't necessarily the best representation of the beer in the first place.

 

It also will ensure that the tried-and-true world class beers are rewarded for consistently being fantastic rather than just a fad. If a new beer stays consistently great, it will make it on the list. I feel that's how it should be. Nice post!

 

By doing this, we're losing so much more data... At the very least change how the scores are calculated. Weight the number of hads in with the average reviews. Right now you have beers like King Sue (100 rating) far below beers like Hopslam (98 rating) just because of how long it's been around.

Better yet, grandfather in the old ratings without reviews and require reviews from here on out.

Quality of reviews WILL suffer greatly if they are required to count towards the ranking. I'll bet most people won't take the time to write anything close to a helpful review and will just be going for that 150 word count.

 

Well, BA existed successfully for years without having only ratings and then survived a change from requiring reviews with lots of words to allowing quick ratings only, I suspect it will survive a move back to asking people to share their sensory experience in 120 words or more as background context for their ratings.

Also, notice that without your BA friend sharing his sensory experience of a beer with us, his insights don't provide much if any value to those of us who don't know him, especially when those insights are rather well hidden in a hazy cloud of numbers, many of which come from people keying in numbers just to improve their tick count. Even with only 120 words about the experiencing of the beer it's easy to tell the difference between those with insight and those throwing out words to fill space.