New Rating System Feedback

Nailed it

 

If that's the case, then you shouldn't have told me what your bluegill looked like. Rate 'em as best as you can with what is meaningful to others.

 

Maybe for recent reviews. You have to remember that the Hads/Rating (one-score) rating option was only in place for a couple of years (from late 2011). Everything prior was full reviews.

 

This attitude, in my opinion, was becoming the big problem with newer users only using the one-score ratings. Too many 1 and 5 ratings. No beer is perfect. I have never given perfect all 5's for a beer.

 

As was stated by the Bros in the announcement - this isn't a "new system". It's the original system.

Which makes your statement of "people don't like change" ironic. Many reviewers were unhappy that the Hads/Ratings were ever factored into the scoring system in the first place.

Sorry to get all "get off my lawn".

 

Re: "Disabled the ability to rate a place that's just a brewery".

Don't have a prob with this, as I did it perhaps only once. I see my review still counts in my total, but I can no longer see the review. Will that change? If this brewery is upgraded to brewery/bar, will past reviews re-appear?

For beer reviews, I like the return of the attributes, though I had gotten used to giving ratings outside of the 0.25 increments and getting my own final score.... No probs, all in all...

Prosit!!

 

Gotta hand it to you, @Todd : I remember when the rating system was changed to allow quick ratings and whatnot and plenty of pro-review people (myself certainly included) came out of the woodwork to bitch and moan. Now, when a review-positive change occurs, people from the other side of the aisle begin their complain train. You guys are damned no matter what, I guess.

Hey, "casual users"! You don't need to give us a novel. You don't need to pick out every possible characteristic of every possible aspect of every beer. Just take a minute or two out of your day to tell us what you think of a beer and why. If it helps, don't even think of it as "reviewing", but as "telling people what you think". If you've got time to write massive tracts in the forums, oftentimes including greater detail than the best reviews (and even half-hearted statistical analyses!), then you've got plenty of time to just provide your thoughts on each beer's page.

 

The problem I have with this opinion toward 5's on the rating scale is that I am not rating toward how perfect or imperfect a beer is in regards to it's style or compared to others of it's type. I know I speak for a large number of people that use this site, when I say that I rate a beer a 1-5 based on how much I enjoy or like that beer. If I find something that I love, and would buy again every time I come across it, I want to give it a 5. Same goes for a horrible beer, that needs to be marked a 1 because it was horrendous.

In my opinion, there are too many people that have this mentality that you can't rate more than a couple beers, or in this case even just one beer, with a 5. We are rating beer in regards to what we enjoy (otherwise why would we even do it?), everyone has a favorite. We need to stop putting a 5 rating up on a pedestal that no one is allowed to touch. It doesn't make someone a connoisseur or an expert because they have never given a 5. IMO, I start to question their ratings/reviews all together if they think that nothing can get the highest score available.

This is what brings me right back to one of my problems with the new/old rating system. It is trying to deter anyone from giving a 5 or 1. Which I feel does not set a very good tone for the rating system as a whole.

 

Well here's a minority voice from the other side of the issue. A rating tells me that someone might or might not have liked a beer but not why and since people's tastes and level of experience with beer differ somewhat the why is quite useful. For example, living in PA with its case law I am sometimes forced to make a choice between buying a case of beer or letting it sit until someone else buys it or it ages beyond its shelf life (and then I won't buy it). I have learned that there are certain reviewers whose tastes are similar to mine (and I've never had the chance to meet them either).

Since these reviewers explain what they like and do not like about a beer I'm better able to evaluate their rating score and make a decision (as you might guess buying a case can be a pretty expensive mistake). I've done that twice in the last 2-3 months and it has worked out beautifully both times (there were no real surprises in what I encountered in the aroma and flavors of the beer, and at least one of them is now on my permanent rotation list). Call that overthinking if you will, but I'm simply not able to spend $60-80 on 24 drain pours.

 

But it should be. Possible scores range from 1, 1.01, 1.02.... to 4.98, 4.99, 5. Meaning there are 400 different scores a beer can have.

So if you rate a beer a 1, you are saying it's in the bottom .25% of all beers. If you rate a beer a 5, you are saying it's in the top 99.75% of all beers. On average, you should only be giving 2 out of every 400 beers these scores.

I can't seem to find it, but there used to be note directly on the BA reviewing page that stated something to the effect of "All 5's should be considered rare" or something to that effect.

As an example - my highest rated beer is JW Lees Harvest Ale - I gave it a 3.5 / 5 / 5 / 5 / 5 , for a 4.93. I didn't think the smell, taste, or mouthfeel could be improved, but the beer didn't look great - it had very little head, and was a bit muddy looking. Why would I change the appearance score to 5, just to say that I liked the beer a lot?

 

Maybe I missed this but why not have precise ratings for each attribute? I like the idea of more thoughtful ratings and requiring users to score each attribute accomplishes that but doesn't removing the precise function altogether go against that notion?

 

Having quick ratings doesn't change that. Those people whose opinions you trusted still wrote out thoughtful, useful reviews even when quick reviews were an option for them so I'm not sure why you're such an advocate of making everyone switch to this more rigid scoring system. It's not like you're all the sudden going to start paying attention to every new reviewers' opinions/reviews so long as the people you trust are still doing full reviews.

Side note, I'm not against (or for, necessarily) the outliers thing. I don't have a huge problem explaining why I score something so low if it's that terrible. Not sure how I feel about having to explain why I rate something higher than the average score, though. "Because I like it, dammit!" is 5 words, I guess.

 

So you're making a big deal on your 4.93 score vs. someone's quick 5 rating? Seems kinda silly for such a tiny, tiny difference. A beer doesn't automatically shoot to the top of the ratings or get put on a pedestal because a couple raters give it a 5.

 

Good question. Some of it is simply because that's the system I "grew up under" (but in those days reviews had to be much much longer than 5 words and there was no option to do attribute rating without a full review) and I considered the change to ratings only, easy or other wise, a poor choice in the context of this site. But I figured it was worth watching to see what would happen after it was put place and to see how it worked out. Now that I've had a good bit of personal experience with doing both and read the reasons for moving part way back to the old system I'm convinced that what differentiates this site from others, and what makes it particularly valuable, is the combination of the ratings and those reviews. If all someone wants to do is tick off that they've had a beer, a spreadsheet works just fine there's no need to rate and review at all. As I indicated above, numbers tell me something useful, but conceal a lot of the story and create ambiguity. Even 5 words give me much greater context in which to evaluate a score, especially on new beers where there are very few reviewers and the potential effect of homerism is much greater.

As for your particular 5 words you just mentioned, I've read many of your posts (and a few of your reviews) since you joined the site and know quite well that you are much more articulate than that, know much more about beer than that, and could easily give me or others 25 or more words about what was so exceptional for you about the beer that led to the 5. In other words, your opinion matters and simply giving a 5 which is an outlier doesn't share with others what you think.

 

I would think that of the quick ratings, a lot of 5 ratings vs. 4.75 would have an effect, as it's 5% higher score.

But no, the larger issue I have is the frequency of users giving out such high ratings. I see that out of 1100+ ratings, you have given 4 perfect 5's. That seems reasonable, as it's 0.4% of your total ratings. I no not agree with @JLaw55 's stance that any beer you would "buy every time" should have a 5. He's given 10 perfect 5's out of 125 total ratings, which I think is way too frequent.

 

If this was explicitly stated on this website, I have not seen it, and therefore may have misjudged the entire point of this site. As I believe many others have. But if we all kept our ratings and reviews to exactly that sentiment, there would be a lot of users leaving this site. Everyone will interpret the point of this site differently.

Personally, while I somewhat understand what you are saying in regards to 1's & 5's, I still disagree. I think that is way too much of a formulaic approach to beer. Again, everyone will have their own way of rating and reviewing, but this seems way too intense of an approach to me. It's meant to be enjoyed, not broken down by mathematical equations and percentages. At least not to the degree of using a formula to determine how many 1's and 5's you should have.

I do agree with you as well though on changing the look to get a 5, it does not make sense. But I attribute probably 95% of my scoring to taste alone, the rest to smell and mouthful, and very little to look.

Perhaps as someone said before, damned if you do, damned if you don't. The system will not fit everyone's style or preferences. I simply believe it should be a bit more open to interpretation.

 

And I would end up on the other end of that spectrum. I see a problem with having 1100+ reviews and only 4 or 5 perfect 5's. I see that as way too infrequent. I would wonder if some of those beers rated and reviewed, were not taken down a peg or two simply to keep from giving another 5.

There are definitely different schools of thought on the way we all rate our beers.

 

Personally I see a 5 to mean "best of class", not perfect.

If 5 means perfect and therefor is unattainable, then there is no point in even allowing a 5 rating.

Just my opinion, but I see never being willing to rate something 5 and rating to many 5s is two sides of the same coin.

I also think that the idea that only 2 out of 400 ratings should be rated a 5 is based on a false premise. That being that a BA member's rating represents a random sample of beers. But a closer look at this audience tells us, they know more about beer than the average person, and use that knowledge to actively seek out beer they will like. Therefor it should be no surprise that scores are skewed in a positive direction. That said 5's still should be a minority, there are only so many "best of class" beers, even when viewed individually.

As I said previously, I did like the single rating system. But I'm also OK with the changes. I'm adjusting the way I rate to accomodate it. I may rate a little less here, becasue of the added time involved with taking more notes and considering more factors. But I will still rate frequently.

To some degree I see it as BA asking us to "step up our game". I'm OK with that.

One final thought, for those that really want to track their personal single rating, why not think of the "Overall" score as your personal rating and the calculated one as BA's score?