Philosophy of Rating

I think if you look at the beer styles on here, you will see that people tend to not rate within the style, but do rate across styles. Certain beer styles are certainly inferior, at least to the kind of people who care enough about beer to get on this forum. It doesn't seem right to rate PBR (ie, the best AAL) a 5.0, the same as Good Morning or Heady Topper. As long as you rate consistently within that style (ie, you handicap all other wits the same way you handicapped White), then you're doing it right.

 

When I first started reviewing, I would read the style on the Beer Advocate page before submitting my review. Unfortunately, it seems like some information was left out of the style over views (like head retention/lacing for particular styles). I do work hard to try and rate within a style, but I am not perfect. I am very aware of my biases, and try to tag those at the end of a review (i.e. I don't care for brown ales, but I still review them as practice for trying new things and picking up flavors).

As per the site as a whole, the solution to pollution is dilution. If we can work towards keeping great overviews of each style for users to read, than we may increase the number of folks reviewing within a style and not based solely on what the user likes/dislikes or what the can/bottle claims. I'm not saying we're polluted with bad reviews, I'm just saying it could certainly help. It's a heck of a lot more difficult for a beer to get false, over-ratings when there are a thousand reviews vs. one of those with ten or hundred reviews (which is why I also think the top-250 should be limited to beer with over 750 reviews).

 

All styles have exceptional (A+), bad (F), and in between examples, so that's how I believe they should be judged- how good is this beer at what it's trying to be?

People do have different expernences and preferences, and there's no preventing that from entering a review, but it shouldn't be your primary framework when composing one.

My experiences also tell me that there are great pleasures to be had from a beer such as a well crafted Vienna , and they can be as great as any other thing I have run across.

The disparity between the styles, if you want to call it that, is more that your chances of finding a great one in certain styles will be higher than in others. But there will be A+'s in all of them.

 

I mostly rate across rather than within styles. My highest rated witbier (probably Allagash White) won't be as highly rated as the highest DIPA. If it's a style I really don't get, I won't rate it at all.

On the other hand, I do rate a beer assuming it's the sort of beer you're in the mood for. So, for example, I rate beers with deliberately low ABV based on how good they are when you want a low ABV beer. Similarly, I wouldn't rate an imperial stout poorly because I was mowing the lawn in the summer and it was too heavy. I'd rate it based on how good it would be compared to other beer options in its ideal situation.

I suppose some people would say that you should rate a witbier based on the ideal situation that witbiers are your favorite style, but I'm just not able to imagine myself in that situation the way I can imagine enjoying a stout in the winter in front of a fireplace.

 

There are a number of ways to set up ratings systems to eliminate biases, but they typically require a lot of background work creating the rating scales in the first place. Once those styles are in place biases can exist when different objects/things/beers are being rated. Neither is true for this site. The rating system is borrowed from a pretty good source on the assumption that their work is relevent to ratings here. Similarly no corrections for individual or style bias are applied.

So in the case of this site there are clearly style biases among the raters in general but since those are not controlled for (this is a consumer oriented site not a basic research project), ignore comparisons across styles.

Each style should be treated on it's own as a separate thing and this is where paying attention to style comes into play. If a particular style involves wheat and/or hops used a particular way, there should be lots of head and retention. However, for other styles, in part because of higher ABV there won't be much head to speak of when comparing across styles. Thus an excellent RIS will not have as much of a head as an excellent Hefeweizen. So the RIS should be considered relative to its style rather than compared to the Hefeweison when evaluating its head, lacing and head retention. (etc., etc.)

So rate each beer as best you are able when compared to others you have had in the same style. If you like certain styles more than others you'll probably give them higher scores acoss the board than a style you don't like but that doesn't really matter once your realize that a numerical score of 4.2 on an RIS doesn't have the same meaning as a score of 4.2 on an Hefeweizen (too many differences between the styles). Similarly a beer on the top 250 list is basically a popular beer not one that is better than others not on the list.

Bottom line, rate within style as much as possible and if you truely don't like a particular style don't rate it at all. Similarly if you are looking for good examples of a particular style look at the top beers within that particular style for ones you haven't tried.

 

I think it's the responsibility of the reviewer to rate to style. I'm certain that eventually, someone will make an AAL that earns a 5. I'm yet to encounter it though. It's vastly unfair to not rate to style. Using myself as an example, I don't care for Scotch Ales, I never buy one for my own enjoyment. But, since I'm something of a ticker, if I see one I haven't had I'll try it. And, because I don't like the style, I put extra effort into not letting biases slip in. By the same token, i really love Tripels, and have to be careful not to give a beer too high of a score just because I love the style.

I think folks really need to approach each thing on its own terms. You wouldn't order salmon at a restaurant, then say how terrible your steak was, why do it with beer. The fact that people do review with their biases in place, is a major reason why the ratings and scores don't matter to me, and why I don't look at them. I think fairness is an important trait in life, and that we should try to do the fairest thing at all times.

 

In business and engineering and other fields they use "performance metrics" or just "metrics" depending on what you want to describe. The idea is, if you want to make consistent judgments then your criteria for those judgments need to be explicitly defined, and you need to execute your assessment of those criteria in a repeatable manner from one judgment to the next. What is perhaps the most difficult with beer, as the OP pointed out, is the varying styles. Clearly you need a new set of metrics for each style, because the criteria of a "good" IPA are worlds away from the criteria of a "good" stout, for example. Of course, subjectivity of different persons comes into play, but that's a given going into any rating system; so if you want true accuracy the best you can hope to do is be consistent in your own ratings, and build up a significant enough database to see a trend in your own preferences. I always told myself that if I decided to start rating beers I would write up a thorough set of rules and metrics for what each criterion really means and how it should be scored. I don't rate, however, because I fear I'd get caught up in the rating and lose some of the beer enjoyment.

 

Yeah, personal preferences will certainly come into play, they almost must. Folks hopefully stay aware of it though. Just because someone prefers Mexican cuisine over Salvadoran, doesn't mean they can't appreciate the Salvadoran for what it is; similar, but still different.

 

One reason for this is that brewers are concentrating on certain styles. There are simply more brewers trying to make knockout IPAs compared to knockout Vienna lagers.

Aside from that, even though I can completely sympathize with and understand the position you've taken above, please don't mistake it for a universal truth. One man's transcendent IPA is another man's undrinkable mess. In my case, some of my great drinking moments were with kolsch. I know some would put these beers in your "pleasant on a hot summer day" category... and in some cases they would even feel generous with that. For me, I believe that the bias you have above is related to simplicity. Can something that you consider to be simple be as good as something that you consider to be complex. I believe it can. One perfect note can be better than noise (or even harmony). Please note that I'm not claiming that a Vienna or a Kolsch is inherently simple, or that an IPA or saison is inherently complex.

As far as rating goes, I don't rate beers here so my opinion might not be worth much - but I wouldn't worry too much about it. Rating to style is far from a perfect methodology. I think that one should be aware of their own preferences - and you clearly are. If someone sees this as a problem, then it's one without a perfect solution.

 

At the end of the day, it's all beer. You rate the beer you're drinking against the other beers you have and your own preferences. This isn't a judging competition where adherence to style matters. How do you rate a style the first time you have it? Or even the 2nd, 3rd, 4th, 5th time? If you're drinking an Amber Ale and you think it's average because you like other styles better, then rate it average. Someone else will come along and rate it higher because Amber is their favorite style. This allows not only the better beers within a style to rise to the top, but it also allows the better beers overall to rise to the top. And when I say "better", I mean beers that a larger portion of the BA population finds to be particularly appealing.

 

People tend to over think this. The great majority that review are just enthusiasts that have an opinion, and like to share that opinion of something they drank on a free forum open to anybody that wants to read it. As long as you are honest about your sensory experience (Appearance, Smell, Taste, Mouthfeel, Overall) then the rest takes care of itself. Everybody has a bias of some sort, and that can be easily articulated in the review so others can be aware. I really think thats all there is.

 

I agree completely. Also, I think the "inferior" and "superior" styles argument is not helpful in determining what truly is good beer.

 

Please, all BAs, force yourself to rate within style. This is the only way we can have a meaningful database. You or I may not like Belgians, but someone else does. "Style" tainted reviews diminish the value of all our reviews (statistically speaking).

After coming on awfully preachy above (one might say dickish), I will admit that I am absolutely guilty of violating this, especially when I was new to reviews, but sometimes even now I find catch myself with style bias.

 

Beers from any style can be good and highly rated. I rate based on how much I like the appearance, aroma, taste, and mouthfeel, and usually consider the style in my overall rating, which is also what I think about the beer's drinkability and overall experience. If you rate something a 5 because it is the "best example of the style", how do you know there won't be a better example in the future? It should be rated by how much you like something imo, which ends up being subjective, along with some consideration of the style as well.

 

The thing is, though, that rating "with style in mind" is much more relaxed than strictly rating "to style", and doesn't really require all that much overthinking; just a slightly different mindset going in.