New Rating System +/-

no, that is what the overall (*and* drinkability!) score is (edit: was) for, as todd has explained on numerous occasions in forum posts.

appearance, smell, taste, mouthfeel:
grade to style as best you can, obviously some subjectivity.
overall: your personal overall enjoyment of the beer (100% subjective).

for some reason a lot of people didn't get this (often *insisted on not getting it* because they didn't like it), but this was very clearly spelled out to be the system. we lose this entirely without categories.

i mean, it bears noting that my reviews (and anyone else's who actually read todd's guidelines and posts) are numerically meaningless after removing categories. i don't think it's a huge deal, but there you go. (edit: "meaningless" is too strong, but they won't accord well with the writing, and they absolutely are not what i would choose if the number meant "overall enjoyment" not "aggregate of my enjoyment and my perception of how well the beer was made").

 

OK, I finally decided to comment on this, and add my own personal thoughts for someone who has thousands of reviews on here.

First positive... I love the new up/down voting of reviews. It will definitely help weed out the people that don't seem to know what they are doing, or are bashing/trying to bring up a rating for their own personal gain. I feel this will be a big improvement for the rating system.

Now the negative... Taking away the precision for each individual category is not great. One of the big things that made this site great in the past was the ability to really see user's reviews and content and specific numbers for various components of a beer. It really helps to dissect a beer and really get an idea how well it was made, and appreciate the care taken in producing the beer. Just throwing a number up for a beer hardly does it or the brewery justice, and really doesn't help others trying to decide if they want to try that beer or not.

I have many friends and acquaintances that specifically decide what beer to get in a bar or store based on my (and others') reviews. Only having a basic number and/or small summary doesn't help a lot. The really tough part to swallow though is that all of my thousands of reviews from the past are now diluted since the individual categories are gone. I won't be able to go back and compare notes on what I thought for each specific category for beers I've had in the past and want to see how they have improved. That's a little disturbing, and not sure why at least the history couldn't have stayed.

Advocating for craft beer requires the feedback in order to enhance the entire culture of it. A brewery can't really get better if they don't have feedback for their beer, specifically what was bad not just that it was bad. Someone gives the beer a low rating, I would like to know why, and I'm sure the brewery would as well. Being able to give my feedback about a specific beer, and have the brewer comment back to me is what really made this site great, and I would hate to see that go away.

Glad to see that you guys keep trying to improve the site in general, just think this specific change was not a great enhancement.

 

Thanks for the pure logic bent on what was a pretty straightforward attempt to make simple points.

I guess, looking from a wide enough, and unrelated to topic enough, angle there can be a reason that is a wrong reason, such as mistakenly using a product for something it cannot do. OTOH your point about moral order is misplaced, as there is no moral order regimen one can, nor need to, apply to ones own personal preferences. In this case the opinions of others, i.e. moral order, are irrelevant. Indeed that was the point.

As for clicks I don't believe there is a certain level of payment reserved for some clicks, a different one for others. That of course is the only relevant thing expressed by my statement.

As always it is fun to see you inject your knowledge of logic to situations in which it is either misplaced, or obscures, in this case both.

 

the bros need clicks for business. some clicks are better for business than others. that's totally relevant.

 

Which clicks are better than others? Or is every click, from veteran or noob, equal, as I expressed as my belief?

 

the ones that bring more clicks, and the ones that come back to click again. also, the ones that add content (which is covered by brings more clicks of course).

 

True, there are some that aren't acknowledging it, but at the same time, I think most people's problem is that while the change didn't take away the most important part of the reviews, it did take away something without really providing any benefit on top of what was already there. They (and I) are having a hard time seeing the benefits of the change.

I may not have done a good job of articulating the problem I'm getting at. My point (and I think it's one that @markdrinksbeer mentioned as well) is that it's simply not possible for people to actually differentiate between beers on 401 different data points (1.00 - 5.00 in increments of 1/100th). The rAVG didn't have this problem because the user isn't differentiating on 401 data points, they are differentiating on 17 of them. The only reason a second decimal point was even necessary was due to the need to keep the significant digits intact, since some of the data points used two decimal places (those ending in .25 and .75). It's not about the "noise" of the resulting scores, but about matching the level of precision in the scores the user can choose with the level of precision that their palate is actually capable of distinguishing. It's reasonable to think that most people can differentiate between 17 data points. It's not reasonable to think they can differentiate on 401.

In the original post explaining the changes to the rating system: http://www.beeradvocate.com/community/threads/update-beeradvocate-rating-system.203219/

From the post (emphasis mine):

"To support the new system, we've also launched two new features that gives users a say in what content matters to them and to encourage more people to review.

Usefulness

• Users can now up or down vote reviews (doesn't apply to ratings), allowing more thoughtful reviews to rise to the top.
• Usefulness has been added as a sorting option and acts as the new default when viewing ratings.
• Beer Karma: Usefulness is being tracked at the user level and will eventually be applied.

Flagging

• We brought back the ability to quickly flag inappropriate reviews (doesn't apply to ratings) for review.
• Flagging a review should only occur if it's inappropriate, not simply because you disagree with it. If you don't agree with a review ... down vote it."

As you can see, they specifically encourage downvoting a review due to disagreement. Whether or not people will actually use it that way is another thing. But the example you provided--"Mmm, good beer"--that, to me, is a review that should be flagged for removal, as it's not a review in any sense of the word. Someone noted that Todd's and Jason's reviews cannot be downvoted, which I have no problem with. I can certainly see how people would abuse that ability based on disagreements or personal issues with them rather than based on the merit of the review. It just seems that the same logic probably applies to everyone--it's too easy for people to abuse the downvote feature, and having a downvote doesn't really add anything extra over only having an upvote option. If the point is to differentiate good from bad reviews, upvoting the good ones will still do that.

I like this idea as well.

 

Look ,dude, either every click equals every other click or it doesn't. What say you? Are some clicks counted as 1.1 clicks, some clicks counted as .9 clicks or is a click a click? Really man, come on.

 

yes, but this doesn't matter. there's no reason to bring up what you're bringing up, unless you're continuing to do the "i'm empathizing with businesspeople" thing. if you're empathizing with business decisions, talk about stuff that matters, not pointless stuff.

you're saying cater to noobs because their clicks are just as good as old timers' clicks. you're explicitly bringing up the good or badness of a click. and the truth is, not all clicks are equal in terms of value. some have a future. some enhance the site. if you wanted to rate clicks, it would be foolish to just cut the site up into noobs and oldtimers, but you *could* rate clicks, and you *could* cut the site up. in fact, if you were changing the forum around for business reasons, you *ought* to do this to achieve your goal.

 

No actually it is the only thing that matters in this discussion.

 

I disagree, instead of seeing a 1.98 that didn't really seem to connect with the review, you'd see a 2.00 in taste that didn't really connect with the review. I don't see that as any more useful. The problem with this review isn't the lack of categories, the problem is that it's a review of a beer they didn't like in which it wasn't

So you don't think you should be able to look at your list and see that beers you enjoyed more have higher scores than beers you enjoyed less?

I think the smell of a beer is important. I don't think it universally contributes 20% to my overall enjoyment of it. For some beers, the smell significantly enhanced my enjoyment of it. For some beers, the smell detracted. For others, I didn't smell much of anything. That didn't actually make me enjoy the beer less and I think having to struggle to care about the smell made my review worse, not better.

I think drinking beer is a holistic experience. That a beer should have poor marks because it wouldn't make a good air freshener while you stick your nose over it and inhale is silly.

Ahh, missed that, I disagree with that as well. You should downvote a review because it's done poorly, not because you disagree.
I've flagged one so far: it was a 1.00 that the review stated explicitly did not accurately reflect the beer, but he wanted to counteract higher reviews from other people that he didn't think were justified. I'd also note that review was from prior to this change - there were still terrible reviews back then.

I understand you completely, I just don't think it's meaningful. The uncertainty between a 4.02 and a 4.01 is indeed less than the uncertainty between 4.25 and 4.00, but it is still proportional. The difference between 4.15 and 3.9 is the same as the difference between 4.25 and 4.00.

 

1) yes, of course i think some beers i enjoyed less should be higher on my list of reviewed beers than some other beers i enjoyed more. this is almost guaranteed by the reviewing guidelines, which i've used for 5 years now.

2) aroma makes a huge difference in my enjoyment of the beer. we would never agree on the weighting of categories, but again, the idea was to try to objectively score a beer (tinged with some smaller % of your own fully subjective enjoyment) according to some general standard. in other words, i don't tacitly acknowledge that smell is 20% of my enjoyment if i review with smell weighed at 20%. i acknowledge that the site is weighing smell that way, and i'm contributing my evaluation of the smell to be weighed at 20% for the average score of the beer.

3) you never had to write about the smell (or appearance), you just had to use it to try to contribute an objective review. you've always been free to write "by the way, i don't care about smell. that's why i didn't factor it into my Overall score."

you're arguing for a drastically different system than has been in place for over 10 years. you're also arguing that because you don't like the way the system worked for those years, we should retroactively change the meaning of every review according to your new preferred way. how is that defensible?

 

Regarding 'clicks' and traffic, one would think that a well crafted and thought out beer review would get more people looking at it (and potentially buying it) than a 4 word review would, no?

 

Well said, I think this is the biggest issue.

 

But under the old system, no one was arbitrarily selecting 4.15 or 3.9. Those scores were the result of calculating a weighted average based on people selecting from the same 17 data points along 5 categories. The scoring didn't ask any user to try being precise to an unachievable level.

The methodology was consistent and normalized across all users. That is no longer the case.

 

Your post is so amazing . I can only say I disagree with one point. The upvoting feature. That makes reviewing a popularity contest. It puts the review into the hands of the mass mentality rather than in the timeliness of the review. I can click on top raters and find all the top raters reviews BUT it makes no sense to upvote a review from 10 years ago when new people are doing new reviews right now. It's not that I don't think that beerchitect's reviews are special, I do. I just think that on this site, for the sake of accuracy and so that we don't start all following the same few people like mini cults, that all reviews should be seen in the order in which they are posted. It's more accurate and allows the new people to be seen in their own right as reviewers.

 

Interesting point. It reminds me of something I have wished for...reviews at the top, ahead of any mere "ratings". I hate scrolling through three or four pages to find 2 reviews. I see the aggregate rating, I don't give a crap what any one person rated it without an accompanying review. I would like them sorted from most recent like you suggest, but also include the number of "thumbs up". Amazon and others do this, allowing people to say "this review was helpful".

 

I think it would be nice if statistics were gathered on this site in a more employable fashion. Like for instance I wish I could pull up the IPA's I have drunk form the category list _ like they do on the other site if you pay the premium membership. That's the other thing that I am hoping all will consider. This site is state of the art, the forums have unequalled quality pariticipation and it's 100% free. What makes this site for me is forums and the participation IN ADDITION to the quality of the ratings. To encourage upvoting for ratings is to diminish that quality by censoring all the other people in a way that may seem small right now but could be measurable over time.

 

since there's a lot of general feedback in this thread, i'll throw in my thoughts on upvoting:

i'm fine with it. i see arguments for and against, but given that there are so many ways to organize review info, i think making default presentation a popularity contest isn't a bad thing.

i mean, suppose a bunch of homers write extremely favorable reviews, and get upvoted by several other homers. if there's *so many homers* that the beer has hundreds of good reviews, that's not "homerism" it's *actual popularity*. if there aren't enough homer reviews but somehow a 5 review gets voted to the top of a beer rated 2 or below, the reader can still see the average score. why not have votes?