Understanding the communities understanding of the 5 point system

Discussion in 'BeerAdvocate Talk' started by stevepat, Sep 10, 2018.

  1. stevepat

    stevepat Disciple (331) Mar 12, 2013 California
    Trader

    Recently I have been making a conscious effort to be more judicious in my ratings. I notice a trend in the broader society to reduce the 5 star system to nothing but 1-star 'This -thing- SUCKZZ!!' and 5-star 'OMG this was the greatest -thing- EVA!!!' dichotomy. Some people add a 4- or 3- star middle ground that usually means it was decent. I notice that I tend toward this extremism myself here on BA, although the community in general seems to be a lot better about it. So I guess I'm asking if the folks here (who tend to be more involved and invested in honest ratings) could elaborate for me a bit on what they think the various levels mean.

    For me a 5-star beer in this system is a true whale, maybe there are 10 or less in any given style available in the world.
    4.5 indicates that the beer (or characteristic of a beer) in question is as good as any reasonable person could ask for. This can sort of bleed down toward 4 depending on my mood.
    3.75 is starting to get toward average but still meant to indicate that I think it is well worth experiencing.
    Once I get down toward 3.5 and lower I am very much writing the beer (or characteristic) off.

    How far off am I from the norm? from the intention of the BA scale? At what 'taste' number are you reaching undrinkable territory? (if I gave a beer a 3 for taste it definitely means I will never buy it again and would likely pass on if offered free)
     
    thebeers and FBarber like this.
  2. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    I'm probably not the best source since I rarely rate anything. As a buyer however and curious about the system I'd say a beer rated as a 3 would be deceidedly mediocre. I'd look at a beer rated 4 and I'd say it was a good beer, worth checking out. You get up to 4.25 and you know it's a most excellent beer, you start in that 4.5 range and it's world class and well worth the effort to get it. But as always its also known the scale is severely tilted towards IPAs and BA Stouts, so the ratings have to be looked at as to style and adjust the way we view it. A 3 for a Pils might very well be a fantastic beer, but an IPA rated the same would be a no buy, at least thats the way I view it.
     
    #2 nc41, Sep 10, 2018
    Last edited: Sep 10, 2018
  3. bbtkd

    bbtkd Poo-Bah (2,058) Sep 20, 2015 South Dakota
    Premium

    I generally follow this. There are only two beers of 589 that I've reviewed over 4.5; Founders CBS and Prairie Pirate Bomb. If I really like a beer and would buy it again I will rate it over 4, and generally rate beers I will never buy again below 4. Beers below 3.5 I would pass up under any circumstances.
     
    brutalfarce and nc41 like this.
  4. jvgoor3786

    jvgoor3786 Meyvn (1,461) May 28, 2015 Arkansas
    Premium Trader

    A rating should have nothing to do with the rarity of the beer.
     
  5. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    So do you look at the rating system to be fairly done if your considering whether the beer is a Dipa or a Pils? There's probably a ton of great lagers that don't hit 3.5. I think the system is severely slanted, I trust it for IPAs and Stouts and basically ignore it for Pils and such. I've only rated 112 beers and I rated 8 at 4.5 or above, no 5s, and some of those I'd re rated them and drop their score a bit.
     
  6. bbtkd

    bbtkd Poo-Bah (2,058) Sep 20, 2015 South Dakota
    Premium

    I tend to find lagers a bit more bland than the dark ales I prefer, and it is challenging to fairly review and score styles you don't care for. Still, I try to score on the same scale so a lager would need to be pretty good to earn a rating over 3.5 from me, same as a stout.
     
    Steve_Studnuts and nc41 like this.
  7. Scott17Taylor

    Scott17Taylor Meyvn (1,332) Oct 28, 2013 Iowa
    Trader

    Here’s my system
    1-3 anywhere from horrible to just not something I would want to drink again
    3-3.5 not bad, I would drink it again, but I’m not going to buy it
    3.5-4 meets my expectations for what the beer should be. Most of my ratings in the higher 3s are beers that are very good, but just in styles that aren’t the BA stouts, sours and IPAs that dominate the top of my and most people’s ratings. I’ve rated a few really good pilsners and marzens in that 3.75-4 range for example. I know I should probably rate to style, but I just tend not to since the BA stouts, sours and IPAs tend to have more mind blowing beers in the style.
    4-4.25 this Is where the beers that really stand out start. Usually if I rate over a 4 it either means it’s a great example of the style or that it just stands out as particularly good.
    4.25-4.5 beers that are very very good, but that aren’t quite among my favorites. Usually these are beers that are really really good, but not quite the ones that blow my mind and make me stop and really appreciate it as better than anything I’ll probably come across for a while.
    4.5-5 these are obviously the elite beers, we all understand why we rate beers on the highest end.
     
  8. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    So I'd trust your rating because you rate to style, but in general it appears a good many are biased. If I see someone who rates something like Budweiser for example rated at a 1.5 I know it's biased. At a minimum to style how could it be much worse than maybe a 3 across the board? Rated to style it should be rated much higher I'd suppose. Just an example of what confuses me a bit.
     
    TrojanRB, FBarber and bbtkd like this.
  9. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    Funny enough the Bud Lime and Orange crap is rated higher than Budweiser by a good bit. Buds rating is like 2.3 or so, I was just curious so I took a peek.
     
    FatBoyGotSwagger likes this.
  10. StoutElk_92

    StoutElk_92 Poo-Bah (1,942) Oct 30, 2015 Massachusetts

    For me personally 5 is about as good as it gets. 4.75 is really great but could be a little better. 4.5 is really good. 4.25 is good. 4 is pretty good, not bad. 3.75 is pretty average, not very good. 3.5 - 3 is meh in different degrees. under a 3 and you might as well not be brewing the beer as far as I'm concerned. Just my opinion though. :stuck_out_tongue:

    *I should probably say though that looking back sometimes I like a beer more that I've rated lower than another beer, so sometimes I might just really like a beer, but because it doesn't have a mouthfeel that I like as much, or didn't have as great of an aroma, my overall rating might be lower. Generally speaking though if a beer is close to a 5 rating from me then I'm loving it. Or even a 4.75 or 5 in an individual rating, not overall, because some beers I love but they might not hit a 5 on all parts for me. I try to be objective with individual ratings. I might not think a beer looks smells or feels or even tastes a 5 but might still really love it. I try to also rate based on how it may be for the style in some respect.
     
    #10 StoutElk_92, Sep 10, 2018
    Last edited: Sep 10, 2018
  11. stevepat

    stevepat Disciple (331) Mar 12, 2013 California
    Trader

    I'm not talking about rarity there. Just saying that my understanding of a 5 rating is that it indicates a very very elite beer, meaning (to me) that there would be 10 or so beers in any given style that achieve that level of greatness. It's possible that one of these beers might be not uncommon but there shouldn't (in my opinion) be lots of examples of beers that achieve a 5
     
    rodbeermunch and jvgoor3786 like this.
  12. stevepat

    stevepat Disciple (331) Mar 12, 2013 California
    Trader

    Also, I just looked at a good handfull of your ratings to see where you were coming from and realized that you seem to always rate all aspects as the identical number, i.e. look:4 smell:4 taste:4 feel:4 overall:4. This seems really unusual to me as most beers I've encountered have aspects that are better and/or worse than others. Could you explain why you choose to rate that way and what the numbers actually mean to you?
     
  13. beertunes

    beertunes Poo-Bah (5,531) Sep 24, 2007 Washington
    Trader

    For me, 3-3.25 is your average, enjoyable beer. Below that, and the beer is likely flawed.

    3.5 is a very good beer.

    4.0 is excellent .

    Above 4 is very rarified air.

    4.5 is elite.

    Only 2 brews have ever gotten all 5s.
     
  14. Celtics76

    Celtics76 Defender (656) Sep 5, 2011 Rhode Island

    In a 5 point system, 3 is generally considered average. It seems like some people here consider 3.5 or so to be average, which isn't correct. I think Todd discussed this in another thread a few months back.
     
    FBarber and Mothergoose03 like this.
  15. FBarber

    FBarber Poo-Bah (2,429) Mar 5, 2016 Illinois
    Premium Trader

    For me, the number matters far less than the review itself. But as numbers go - since you asked about the aspect - Anything below a 3.5 tends to be a forgettable beer. I know that a 3 is average, but in terms of my personal experience with the average rating of the beer and how that correlates to whether I like it or not, thats where I am. Anything in the range of 3.5 to 4 is solidly above average and generally a beer I'd drink again. 4 to 4.5 is outstanding and well worth drinking and then anything above a 4.5 is rarified air. Since we can't really know the reasoning behind people's ratings, unless there are a significant number of ratings on the beer, I tend to ignore other peoples ratings unless there is some explanation (a review) of it.
    I completely agree that lager scores are skewed low here - thats one of the reasons that I take the ratings with a large grain of salt. That being said, I think the inverse is true - IPAs and stout ratings are inflated here - so again, I'm taking those ratings with a grain of salt. If Im looking for feedback on whether I'd likely like a beer or not I read through the reviews and try to glean information thats helpful to making an informed decision.
     
    Mothergoose03 and nc41 like this.
  16. Todd

    Todd Founder (5,603) Aug 23, 1996 California
    Staff

    johnInLA and Celtics76 like this.
  17. jhavs

    jhavs Poo-Bah (1,559) Apr 16, 2015 New Hampshire
    Premium Trader

    I always equate the scale to school grades, which may not necessarily be how the scale was designed. In my view, a 4.0 is an 80%. In school some people were happy with this grade, some were not.
    I grade a beer that I think is good (I would drink and buy again if seen) a 4.25.
    I grade beers I really like and would seek out a 4.5.
    Above a 4.5 is really special.
    4.75 and above truly set themselves apart from the pack.

    3.5 - 3.75 in my personal scale is something that I am fine with, but I probably wouldn't buy or drink again if I had better options available.
     
    warrendietrich2001, nc41 and mkh012 like this.
  18. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    Let me also add there are a few reviewers that I know rate to style, sometimes I just zero in on a few BAs. I trust their input.
     
  19. jvgoor3786

    jvgoor3786 Meyvn (1,461) May 28, 2015 Arkansas
    Premium Trader

    That makes more sense to me. I guess I misunderstood what you were saying.

    I agree that rating each aspect differently would be more accurate. I started because I was unsure of my ability to rate each aspect. As I got more accustomed to rating, I did it for consistency with my earlier ratings (and to keep ratings the same as the other site I use to track beers). Plus, it's a bit of laziness on my part. If I'm not reviewing the beer, I don't always take enough time and thought to rate each aspect. I guess, in the end, it's just my preference.

    As far as my scale, I try to rate somewhat to style, but in reality, I don't think anyone does that. If we did, there would probably be AALs and Malt Liquors rated near a 5. For me,
    4.75-5 is an incredible, world class beer.
    4.25-4.5 is well above average, great flavor, feel, and smell, and something I'll regularly buy and drink.
    3.75-4.0 is average to slightly above and something I'd drink again.
    3.25-3.5 is average to slightly below average and something I'd probably not buy again.
    3.0 and below I avoid and didn't like.

    My ratings tend more toward my tastes than perfectly to style, but I'm guessing that's the same for most people. Also, purists (and the founder of this site) will say a 3.0 should be the average rating on this site, but that doesn't seem happen in practicality here

    Thanks for taking the time to respond to my post. I hope this helps
     
    jhavs likes this.
  20. Ceddd99

    Ceddd99 Initiate (86) May 14, 2018 Michigan
    Trader

    How could it be worse? Because of individual tastes. You can't necessarily assume someone is biased just because they give a low rating to a classic beer. For example, I hate Labatt Blue. I'd give it a 1.5, but it's a popular beer where I live. Budweiser I would probably give around a 3 like you said. I actually like the flavor of Budweiser but to me it has an unpleasant aftertaste that ruins it. That might be a deal breaker for some people, I mean even more so than it is for me. Not that I'd be drinking much Budweiser were it not for that, but I might be more inclined to buy a single can here and there for nostalgia sake as it was one of the first beers I ever drank.

    Another beer I hate is Sam Adams Winter Lager. To me it's one of the grossest beers I've ever tasted even though it has a good rating on beer advocate and I like other Sam Adams beers.

    Anyway I agree with you that in many cases people giving American Adjunct lagers 1.5s may just be people who hate American Adjunct Lagers but without checking their other reviews there is no way to know that for certain.
     
    nc41 likes this.
  21. Ceddd99

    Ceddd99 Initiate (86) May 14, 2018 Michigan
    Trader

    That's the problem with rating to style. On the one hand, it doesn't make sense that IPAs for example, should score higher than other styles of beer just because they are IPAs. On the other hand, I see why people wouldn't think an AAL belonged near the top of the top beer list. Although that raises the question of what exactly constitutes a style? For instance, AAL could be grouped in the larger category of pale lagers, in which case there are some pale lagers which warrant a high rating.

    It seems like people don't want to give as high ratings to beers outside their preferred styles. For example, a stout lover might give his favorite stout a 5 whereas he will only give a Vienna lager a 3.9 even if it's by far the best Vienna lager he's ever tasted.

    Perhaps beer advocate should grade beers on a curve, so the rating is expressed relative to the ratings received by other beers of that same style. The overall ratings could remain as they are, but each beer would receive a second, "within style" rating that could show up along with the overall rating.

    One thing I do from time to time is click on a particular style and then look at the highest rated beers. The problem with this method is that there doesn't seem to be a minimum number of ratings in order for beers to qualify, so rather than being able to view a straight list of the highest rated beers for a particular style, you are stuck scrolling through tons of beers which have only one, two, or in many cases zero ratings.

    I think creating a second, relative rating is better than asking people to "rate to style" because while comparing beers of the same style certainly makes sense, on some level you can't help but also compare each beer you try to all other beers you've tried. When people really like a beer, they want their rating to express that, and not just imply they think it's good for the style.
     
    #21 Ceddd99, Sep 14, 2018
    Last edited: Sep 14, 2018
    Squire likes this.
  22. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    Of course there's individual preferences, but if you look at a standard AAL like Miller or Bud it shouldn't have an overall rating of say 2. It might not be your favorite but it's certinly at least to a mid point for the style. It's certinly not swill and the ratings should at least tell you to style it's average. But if you look at the ratings for Pils and lagers you can certinly see a trend where it appears they are being downgraded because they're not PA or IPAs.
     
    TrojanRB and Squire like this.
  23. readyski

    readyski Aspirant (226) Jun 4, 2005 California
    Trader

    Regarding the lager point skew, I think you just have to apply the correct filter. Use the 1.33 multiplier (3 x 1.33 = 4)
     
  24. NeroFiddled

    NeroFiddled Poo-Bah (10,275) Jul 8, 2002 Pennsylvania
    Premium Trader

    When I write a review I keep two things in mind: is it to style? and how does it match up world-wide as an average to better, or lesser, beer?

    Regarding style, not every beer really fits a style, at least along BJCP lines, and thus a lot of beers need to be reviewed hedonistically. BUT even when I'm reviewing based on style there is that certain je ne sais quoi that comes into play. Certain beers might hit all of the points on the style guidelines, and be well made, but still not have that special magic to them.

    As far as rating on a world-wide scale you really have to taste some of those beers from small countries to know what's going on. Budweiser is a great example in my mind. It is what it is, we're all aware, but compared to some of the beers from south America and China and Russia and Italy and Greece, ETC. that are roughly in the same vein Budweiser is clearly tops in quality. A lot of these other beers have flaws to them. Should Bud not be better than a 3 then? It's not average, it's much better than average. And that's certainly true within it's specific category of American adjunct lager as well.
     
    Mothergoose03, FBarber and Squire like this.
  25. dvmin98

    dvmin98 Site Editor (2,704) Nov 1, 2010 North Carolina
    Premium Trader

    I tend to base my ratings on the style of beer. For instance, Copper from Olde Meck is definitely not a 5 based on overall beers, but based on Altbiers, its a 5.
     
    AyatollahGold and dennis3951 like this.
  26. Squire

    Squire Poo-Bah (2,005) Jul 16, 2015 Mississippi
    Premium Trader

    Quite frankly my reviews are influenced by how much I like the beer being mindful of whether or not I would buy it again.
     
    Premo88 likes this.
  27. EvenMoreJesus

    EvenMoreJesus Champion (829) Jun 8, 2017 Pennsylvania
    Premium Trader

    And that's why rating to style is important. You can't compare apples to oranges, so why should you be made to compare Altbier or Pils to NEIPA or Imperial Stout?
     
  28. rodbeermunch

    rodbeermunch Poo-Bah (4,215) Nov 26, 2015 Nevada
    Trader

    I understand your position, but I do factor rarity into my reviews, if ever so slightly, and I mean slightly.

    The more available the beer, the better it fares. If your beer was a 34 bottle batch, it will be impossible to get a perfect 5 from me. If your beer is +$1 per oz, you're going to have higher expectations. If I have to stand in line, you're going to have higher expectations.

    I mean, don't get me wrong, shit like St. Lam and Fou Fune are still ~4.9/5 for me. Put that shit at the click of a button or in Wal Mart or Safeway on the reg, and those will likely jump up to a 5.

    I try to keep in mind rating to style, but I also rate to "my" style, sometimes those are at odds. I'm sure my average lambic score is higher than my average doppelbock score.

    Also, despite the cliche, I believe you can compare apples to oranges. For instance, apples are most frequently red or green in color, whereas oranges, tend to be, well, orange. . . you generally bite into the apple without peeling the skin, whereas most people peel the skin off their orange. Most don't eat the core of the apple, whereas most don't eat the outer skin of the orange.
     
    jakecattleco and jvgoor3786 like this.
  29. Ceddd99

    Ceddd99 Initiate (86) May 14, 2018 Michigan
    Trader

    Yep, you certainly can compare apples to oranges. I think what that saying is trying to get at is that there are different types of comparisons and apples and oranges are the sort where they each have their own advantages and can't be wholly replaced by each other.

    An example of the other type would be the comparison between two brands of flour where one is just inferior in multiple ways and offers no real advantage so you can happily stop buying the other brand.
     
  30. rgordon

    rgordon Champion (850) Apr 26, 2012 North Carolina

    Ratings are over-rated. Being so subjective is both a strength and a weakness. I'll write reviews on occasion, will always take notes, but, for me, rating beers really doesn't matter at all. I'll always be able to figure out what new beers I want to try.
     
  31. drtth

    drtth Poo-Bah (3,634) Nov 25, 2007 Pennsylvania

    You may be missing a feature that will be helpful here.

    If you go to a style page, and, staying on that same page, click the "score" link to sort by highest score, this happens and is the only way I've ever seen a list of beers with only 1 rating.

    However, just below the description of a style there is a link that takes you to a list of the highest rated beers for a particular style that has excluded the beers that don't have the minimum number of ratings to make the "best of" list.
     
    #31 drtth, Sep 15, 2018
    Last edited: Sep 15, 2018
    Mothergoose03 likes this.
  32. drtth

    drtth Poo-Bah (3,634) Nov 25, 2007 Pennsylvania

    This is a critical point in dealing with the role of numerical ratings. Different people will inevitably use the numbers in at least slightly different ways. However, as the number of ratings grow, the average becomes more representative of the perceived reality for the community of raters. The two extremes you mention basically tend to cancel each other out. The garbage ratings tend to get lost in the noise and become a waste of the rater's time to do them in the first place. Stylistic differences will persist under the present system of collecting and displaying ratings, but those differences can be ignored by looking at numerical values only within a particular style and not worrying about cross style comparisons.
     
    #32 drtth, Sep 15, 2018
    Last edited: Sep 15, 2018
  33. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    This is about where I'm at as well for most beer styles. But somewhere out there is the world class AALs to style that's rated at 2.75, so as long as we understand the slant it's useful. I like Michelob could I really rate it at a 4.5? I get AALs are polarizing but a really great Pils should not be, there's certinly world class Pils out there that fairly rate 4.5 plus. Pils are neither bland or brewed with rice, thin or watery. The glitch in the system is us the reviewers not the scale itself. And my early reviews reflect my own bias towards IPAs, so I wasn't using the tools fairly or accurately. So now I just don't, even though I know better Imstill a bit biased, and I catch myself doing what I rail against. It certinly isn't easy.
     
  34. M-14

    M-14 Initiate (118) Feb 5, 2014 California

    5 Would go to Paso Robles to buy

    4.5 Would go out of my way to buy

    4 Would buy again

    3.5 Would try again

    3 Glad I tried

    2.5 Better than water

    2 Should have had water

    1.5 Didn't finish

    1 Drain poor
     
    smbslt and TrojanRB like this.
  35. Squire

    Squire Poo-Bah (2,005) Jul 16, 2015 Mississippi
    Premium Trader

    When I come across a new beer I do not check the ratings before buying. I know enough of styles to identify what's interesting to me and ratings won't change that.
     
  36. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    I don't really rate beers, I might add a few thoughts on a beer for NBS, but in depth reviews I stopped doing years ago. Also agree with squire, I've never used the reviewing system to help me buy beers, I generally just look at the style, and I look for a canning date.
     
    rgordon likes this.
  37. dbrauneis

    dbrauneis Site Editor (7,485) Dec 8, 2007 North Carolina
    Premium Trader

    There are only a couple of styles I check reviews (not scores but descriptive review content) for... American Black Ale and Sour Stouts are the ones that come to mind as they can be really hit or miss for me - and I checked for reviewer's whose palate is similar to my own before purchasing.
     
  38. Scott17Taylor

    Scott17Taylor Meyvn (1,332) Oct 28, 2013 Iowa
    Trader

    I personally do rate with a bit of a style bias, but I rate to how much I enjoyed the beer. I personally rate mostly to keep track of things for myself. The fact that the overall rating helps others decide what to buy is just a nice bonus.
     
    alexsergio, jakecattleco and nc41 like this.
  39. nc41

    nc41 Poo-Bah (1,607) Sep 25, 2008 North Carolina
    Trader

    I feel comfortable rating IPAs and such, I've taken on my share. Perhaps Pils too, my new favorite. But others not so much, and the ones I feel comfortable rating are also time sensitive beers. It's not really all so easy, there's so many variables.
     
    Scott17Taylor likes this.
  40. Scott17Taylor

    Scott17Taylor Meyvn (1,332) Oct 28, 2013 Iowa
    Trader

    The only way to get better is to continue to rate and review beers. Honestly I don’t know why you wouldn’t feel comfortable rating everything you drink, even if you’ve only had a few examples of the style or if it’s not really one of your favorites your opinion is still valid.