Thoughts on the latest updates? Top 250, etc.

To late to edit.

Correction, "150 characters" not "120 words"

 

Agree with this whole heatedly! I rarely come to this page, and I assume most dont either...But maybe they want to "hide" to not deal with "us" who are complaining..

While I totally understand why they are doing this, the bad result of this is punishing small/new/unknown breweries...Beers should have ALL the ratings calculated in..JMO

 

I don't use the top 250 list that often but I do use the top beers in each style lists and this change is terrible for all of them. As people have stated, this change absolutely crushes newer breweries that are doing awesome things. I know I used Trillium as an example before but I just looked at Double Dry Hopped Fort Point because I had that beer recently and it might have been the greatest hoppy beer I've had. It has 420 hads but only 21 reviews. Dropped from like #3 on the top APA list down to #18. Thank god it has more than 18 reviews or it wouldn't even show up on the list. How stupid is that. 420 ratings but only going to count the 21 reviews. What a terrible waste of valuable data. I'm sure there are lots of amazing beers out there that are going to have something like a couple hundred hads and won't even show up on the list because of too few reviews.

 

There are waaaaay more craft beer drinkers now than in the past...You cant compare past to now..JMO

 

Now that I have thought about this change and seen it in action, wiping out at least a couple of years of ratings is an incredibly bad idea. The site allowed ratings without reviews, then changed the rules in the middle of the game. If they want to encourage reviews, then leave all of the old ratings and reviews intact and going forward only count reviews. They also need to make it a lot more obvious that ratings will not count towards a beers score.

This change has made scores for newer breweries completely worthless now. Take a look at Other Half as an example, this page had a lot of great info, now with ratings removed the scores are worthless.

http://www.beeradvocate.com/beer/profile/33510/

Also, Cellarmaker took a similar hit. Looking at this page it looks like a little known brewery with little traffic at the place. When in fact it is making some of the best beer in the country.

http://www.beeradvocate.com/beer/profile/32931/?sort=avgD&view=beers

Thanks BA for making a years worth of me drinking and rating Other Half beers a waste of time as it pertains to this site.

 

To one of the points of my previous post, I have a hard time believing that you've gotten enough experience with a beer to provide a rating that is worth much. Personally, I think culling the scores from these types of experience is a good thing for anyone who wants to use the scores on this site as a guide.

And if you can get a good enough sensor experience from a 4 oz taster to truly evaluate a beer, then write a few words and validate that fact.

Why is an average beer any less deserving of an explanation than a terrible or stellar one? The whole point of a review is to justify your rating. I would think that applies across the board.

To me, a number without context doesn't give much insight. Your friend certainly is very experienced, but I'm of the opinion that writing reviews helps hone the ability of truly evaluating a beer honestly. If you want to just drink and enjoy beer, then more power to you--I absolutely see the value in that and I have found myself in that mindset many times. But those times are when I don't rate or review a beer, and I also don't see any problem with eliminating my scores when I just "ticked" the beer, because I personally don't know how valuable those scores were to the overall score--after all, I was just having a beer.

Look at it this way--as a music theory major in college, one of the parts of getting my degree involved reviewing and evaluating some of the great works to help us understand things about form and composition. But in addition to evaluating those pieces, I would also many times just listen to them because I enjoyed them. It involves two completely different mindsets for experiencing the music and I think a parallel can be drawn between that and the two mindsets of enjoying beer.

Putting some separation between the "just enjoying" approach and the focused evaluation approach is a good thing in my opinion. The one thing I will say that might be interesting given the arguments made in this discussion is having a Reviews Only score and a Ratings Only score. It would be fascinating to compare the two, but I imagine that would be a fairly large level of effort for them to implement such a parallel rating system.

Honestly, if you are only providing a rating, how much time have you actually wasted? I would argue that you've wasted little to no time, as the removal of ratings from the score calculations didn't eradicate your experience of drinking the beers. Literally all that has been erased is the 30 seconds (at most) you took to log in and select a number score.

The main thing that I don't quite get here is the fact that people who haven't spent a lot of time giving their input are very upset about that input being omitted from the scores. I understand the concerns about how the scores have impacted peoples' ability to see new and trendy beers/breweries more than the concern about scores being lost.

 

Fun fact time. I took a look at the top 10 list for APAs and calculated the ratio of hads to reviews. Here is the data along with double dry fort point:



Notice the trend? Even though beers like what is enlightenment and Citra pale ale have fewer 'hads' they have a higher review to had ratio. Trillium consistently has a ratio below 0.1, whereas the average for the rest of the top 10 excluding trillium is 0.15.

What this tells me is that Trillium beers are under-reviewed next to their peers. I can only hype-othesize as to the cause (see what I did there) but I suspect there is an underlying reason

The new system seems to have (I think rightfully) given a spanking to really hyped beers - those that tend to be at bottle shares, where people go bat-shit over them and base their ratings on a few ounces. People when just having a small sample don't tend to share any thoughts since they don't have much to go on / or time at the event. 'Ticker' type beers seem to be the ones taking the hit, and I think that is a good thing.

You say that not including these ratings is a waste of valuable data. I disagree - I don't think the data is valuable; it is just noise.

 

What has been eradicated is making the brewery page with the listing of all of their beers for new breweries far less useful. I use those pages to get an idea of how good a brewery is and try to pinpoint their best beers. The trend over last year or so for these new breweries is around 90% ratings. So before this change, I would see a beers score based on 30 people's opinion, now with new change I am dependent on 1 or two people's ratings

2+ years of people's ratings have been wiped out like they didn't exist due to a bad decision.

 

But now you dont have enough data. Far too many beers now have their scored based on 1 or 2 reviews (see the previously mentioned other half and cellarmaker). So based on BA, you have no idea cellarmaker should be a must stop for any beer drinker in San Fransico. So those people will miss out. Eventually more reviews may come in to fix it, but honestly, there is no guarantee this site will stay popular enough to fix these problems. So now you are going by the review of one person who may have no idea what they are talking about. I prefer a lot of reviews, (>10), but sometimes even more ratings (>60) provide as useful information or more. Now I will be going to untappd to be getting a quick idea of what breweries are worth trying and which beers should be my first priority.

 

Most of the other beers in the top 10 there are beers that have been around for a while. I disagree that Trillium is getting spanked because it is hyped. It is getting hurt because it a newer brewery and for the past few years, BA hasn't required reviews. Those other beers have all been around a longer time so I bet more of the reviews are from when reviews were required. I bet if you looked at the ratios of reviews to hads over just the period when quick ratings were allowed, the ratios would be a lot closer.

 

By "around for a while" in the top 10 you mean the 20% of the slots occupied by Trillium?? And how long do you think Hill Farmstead has been around?

 

Quite possibly true - and if so, in a few months time as more reviews get added for these trillium beers (of which 2 are still in the top 10 mind you) they will move back up the list.

 

@pagriley Have you tried any of these beers? Have you had Trillium? If you had you wouldn't be arguing in favor of the new statistics. If Trillium had started in 2005 they would dominate that list. There is no hype there... Just unbelievable beer.

 

The problem is there is nothing forcing people to write reviews, they can still rate a beer without a review and have it not count.

 

Also, have you had any Trillium? I live in the midwest. I have easy access to beers like Pseudo Sue and Zombie Dust. I wish they were my favorite. I would just drink those. I do still really enjoy both but I put them in a blind taste test with Trillium Fort Point and Trillium destroyed them. No bias or hype there. This is obviously just one example. As others have stated, so many other small breweries that are going to be hurt by this change.

 

In which case they will come to dominate the top 10.

 

Agreed, I don't think most people will even notice the whole "your score doesn't count" thing, which is why I'm not optimistic these ratings will get fixed making this feature not very beneficial to anyone.

 

I agree, that the 'had' data could be used to create a different metric to tell you something about a beer - perhaps a 'trending' score or something like that which isn't the same as the rating, but uses all 'hads' to calculate a metric to tell you if you might want to try a particular beer or brewery. It does seem that there a some gaps at the moment - but I imagine they will fill in very quickly.

I am planning to re-look at the top 250 every week or so over the coming months. I expect there to be a solid month of 'shaking out' followed by a few more months of a lot of movement as people start providing more reviews. Hopefully the ratings attached to the reviews will be a bit more thoughtful. Time will tell!

 

This assumes people will start adding more reviews enough to overcome the years of extra reviews the previous beers have. I think their is a better chance people will either keeping just doing hads (not changing anything) or jump ship to someplace else.

 

But they already did. As I have said in previous posts, the data is just wrong now. You are assuming that everyone is going to start writing reviews. I have hundreds of "ticks" on here and I am not going back and writing reviews. I wouldn't remember it enough to write a good one anyway. So you are losing data from hundreds or thousands of people that you wont get back. I think it is actually much more likely that people just stop using the beer rating section of BA from this change. I am certainly not going to search for a beer, look click on the view hads check box and then manually go through and calculate the real rating of the beer. I will just check it out on Untapped which is a shame because the old ratings on here were MUCH better than Untapped.