Changes: Beer Hads, Full Reviews & Ratings

Sorry I don't have time to read everything, but to prevent trolling hads, especially one person signing up for multiple accounts, you could have a 50 had minimum for that user's scores to factor in to a beer's rating.

 

I have, in fact, done that on a couple occasions. I.e. just 'Had'ed a beer when I really don't have anything noteworthy to say about the beer (typically because it isn't a noteworthy beer).

 

Gotcha. Thanks, Todd!

 

I am not too thrilled with the changes. I did not like the numerous Hads coming up with a review and changing the beers final score. .. no text just a number which is not useful at all. I suggest a way for users to only view actual reviews with content. The why (i.e context) is important.

Next 100 characters minimum. I want more content not less. Tasting notes should be detailed. Less details make it harder for me to discern between those that are accomplished at judging beer (i.e Certified Ciccerone, Master Ciccerone, BJCP etc.) and those that are not.

Lastly, I would really like to be able to toggle between Had reviews and full reviews (the format with more characters - not less) so I can compare the difference in the final score. rDev and the other stats are more useful when I can manipulate the data to compare between people who actually know how to review a beer (people with training, brewing experience, or just have had hundreds and hundreds of beers) as opposed to some 21 year old fan-boy saying Heady Topper was a 5.

 

My first reaction was not positive for adding the hads to the reviews, but I always review the beer, so it did not appear to affect my use of the site. Then I decided to gather some actual data before deciding and was a bit shocked by what I saw.
The bogus "1" and "2" rating phenomenon is very clear for many beers. My percentage of over-rating shot through the roof on some of the beers, and when I looked is was all a result of adding 50% hads with ratings of "1" or "2" to overall decent review scores. As noted and documented above, these are clearly the result of lazy or overly fast use of the hads and not an actual result of wanting to rate the beer.
Check out this local Elysian beer that has less than 50 ratings: Hansel and Gretel. The actual reviews range from 3.0 to 4.08 (with one 2.38 outlier). the hads range from 1 to 4 with *6* 1's or 1.5's, *2* 2's. So 8 of 28 hads have a low rating that is completely outside the range of the scores from the reviews (INCLUDING the outlier). if you drop the big outlier 2.38 review rating which has an rdev of 22% (and undoubtedly much higher if compared only with other reviews), there are total of 16 of 28 that are under or just touching the bottom edge of reviews (3). This really shows that the rating columns between 'reviews' and 'hads' are vastly different numbers.
Even weeding out the "1"s, which are almost certainly a systematic error in use of the 'hads' form, would not be sufficient because ALL the 'hads' ratings are obviously much, much lower for every beer than the 'review' ratings.
After looking at a bunch of my reviews, my confidence in the 'hads' review numbers is near zero. There is no possible usefullness in including them in the total averages, and I personally don't see getting any useful information from even looking at them at all
My strong personal vote is to remove the 'hads' from the review section completely. They are a bad, bad, bad estimator of beer quality and make scanning the actual reviews more difficult.
Edit: I see that Todd addressed the issue of filtering the hads from the view, which is great, but I'm still not convinced that the hads review values are comparable to review scores for many beers.

 

I'm not worried about my own reviews, I'm worried about the average quality of reviews on this site. Interesting, well-written reviews (especially those by BuckeyeNation) were what prompted me to join BA in the first place, and the higher minimum character limit at least helped keep reviews to a certain standard*. I don't want to have to filter through tons of useless reviews to find the good ones, and with this change I could see the "top reviewers" becoming dominated by terse, uninformative reviews.

Maybe criticizing the 100-character limit is the wrong way to go, though. Perhaps a better solution (already mentioned in this thread) could be allowing "liking" of reviews or of reviewers, and allow sorting by that. Ideally, the reviewers who put more care into their reviews will percolate to the top. I would be happy with that approach, and the lower character limit wouldn't be a problem.

(*) Granted, there's a lot of variation in reviews, but just compare the reviews for a beer on BA sorted by "Top Reviewers" with the same beer on the other site sorted by "Top Raters". It's typically a world of difference.

 

I'm in support of combining the two reviews/hads, however I'm not in support of lowering the reviews minimum to 100 characters. If your taking the time to analyze and critique a beer, what's a few more words.

 

Lots of good suggestions here, so far.

I'd like to add that I think the minimum # of reviews/hads for a beer to make a Top List and for a beer to make the Top Beers of Fame list, specifically, should be bumped up. (25 for Top List? 2000 for Top Beers of Fame?)

Also, I like the idea of 'Had', 'Rated', and 'Reviewed'. Or, maybe, have the 'Had' review page drop-down box read...

'Did Not Enjoy/Would Not Drink Again',
'Mediocre/Not Something I'd Seek Out',
'Average/Would Revisit',
'Above-Average/Would Like More Of,'
'Exceptional/One Of The Better Of Its Class'

And, then you could assign reasonable scores for each rating above. Perhaps...

'Did Not Enjoy/Would Not Drink Again', = 1.5
'Mediocre/Not Something I'd Seek Out', = 2.25
'Average/Would Revisit', = 3.0
'Above-Average/Would Like More Of,' = 3.75
'Exceptional/One Of The Better Of Its Class' = 4.5

 

A couple posts of mine disappeared as they were responses to some of the user callouts that got deleted from this thread, so I'm going to try to reproduce the gist, which I think had some independent merit:

Some of the users who were named have all their Hads rated 1. This is arguably a misuse of Hads, but also a consequence of the UI making 1 the simplest rating to give. If the UI defaulted to 3, you'd probably see these users with all 3s. This does seem problematic, although I think the effect is minimal so long as users are consistently giving all their ticks the same rating.

However, some of the users had predominantly 1s with smaller numbers of 4s and 5s. I'd like to suggest this might be okay. Among much of the trading & reviewing community, we've normalized an attitude where it's only to give a 4 to a beer that's "okay, not great, glad I got to try it once, wouldn't seek it out again". I don't see a problem with a contrarian view where people give a 1 to a beer they wouldn't buy again and a 4 or 5 to beers they choose to drink all the time. I hardly think this will be the calamity some people expect, and might lead to greater diversity in the Top Beer lists.

 

We are working on this, thanks for the heads up.

 

I'm ok with adding "hads" to the score but think that full reviews should be weighted more heavily.

A good beer is still going to get good scores and a bad beer will not get undeserved praise. Yes, the lists will get shuffled around but it's not the first time it's happened on the site and probably won't be the last.

In the end, they're just numbers. It's all about enjoying and advocating great beer.

 

Personally I like the change. I have to imagine that a lot of reviewers reverse engineer the individual rankings to get the overall score where they wanted it to be in the first place.

I think the Popular list should have a 100 review minimum vs 10.

 

aside from the jackasses who are trolling beers they can't get by '1'ing, this is a really cool change.

 

Todd - thanks for all the hard work first of all. Consider me as a passive user of the site. I don't review . . I just read reviews from those I trust. Detailed tasting notes are helpful. Maybe allowing a user to pick reviewers that a user trusts would be helpful.

 

Since the forum has implemented "following", perhaps the reviews from people you follow are listed first on the page? Similar to how when you click the "bros" score it lists a few reviews sorted to the top?

 

I am all for Had's being part of the equation but why are they being displayed where the reviews are?
Why not just show a had's score that can be clicked on and show all of the people who have had it there?
I mean come on, just look at the new #1 beer, Heady Topper.
25 listings shown on the first page, 5 reviews.
How is that helping anybody?