Are your beer ratings linear?

Discussion in 'BeerAdvocate Talk' started by MNAle, Jan 9, 2020.

  1. MNAle

    MNAle Poo-Bah (1,657) Sep 6, 2011 Minnesota

    This might be a topic only for statisticians, engineers, and similarly-minded math geeks, but I've realized my ratings are decidedly non-linear.

    Let me explain.

    A linear rating approach would mean each .25 increment in the BA rating scale means pretty much the same thing in perceived "goodness" increase to you. I've realized this is NOT true for me.

    I've found that there is much, much less difference between increments from 3.00 to 4.00 than there is between 4.00 and 5.00. IOW, it takes much more to go from 4 to 4.25 than it does from 3.75 to 4, and more again to go from 4.25 to 4.5, and it is very difficult to get a rating above 4.5 from me. I've never issued a 5 rating (IIRC, anyway... I haven't gone back to examine all my reviews / ratings...)

    The same is true below 3.0 as well, although my personal sample size is small. IOW, a beer has to be really, really awful to fall below 2.5.

    I guess I'm a "tough grader" on the very good and a "easy grader" on the very bad.

    Any comments about my "system"?

    To my topic question, are your ratings linear?
  2. TheIPAHunter

    TheIPAHunter Poo-Bah (2,260) Aug 12, 2007 Pennsylvania
    Society Trader

    I love you, but...

    Your intro screams, I'm one of those three, and I see beer in a more abstract way than the average BA. Sir...
    Sludgeman, drunkenmess and TrojanRB like this.
  3. MNAle

    MNAle Poo-Bah (1,657) Sep 6, 2011 Minnesota

    Well, I am one of those three (an engineer), and as a result, I do notice things like linearity, etc... and maybe most (rational) people don't care about such things.

    Not meant to be exclusive at all; but perhaps a bit of self-questioning "am I doing this right"?
    TheIPAHunter likes this.
  4. TheIPAHunter

    TheIPAHunter Poo-Bah (2,260) Aug 12, 2007 Pennsylvania
    Society Trader

    As I said, I love you. Carry on. I'm sure this will generate some good responses.
  5. bbtkd

    bbtkd Poo-Bah (3,102) Sep 20, 2015 South Dakota
    Society Trader

    I concur with your non-linearity, and I probably use even less of the scale. I rate almost none below 3.5 and few above 4.5. A beer has to really suck to get below a 3 from me. Beers below 4 I may have again if offered it or no other choices, but sure wouldn't buy a pack. Most of the beer I buy packs of and/or cellar are ones I rated over 4.25.

    Most of the beer I buy are singles to review, and I won't generally try one unless it's rated over 4 unless I suspect by the description I might like it anyway. Life is too short to drink beers with a good chance of sucking. I'd rather spend my time and money on highly rated beers less likely to disappoint. That said, there are some brewers where I will try anything they make, such as Prairie, Founders, Avery, Drekker, White Elm, Surly, Central Waters, GI BCBS, and Boulevard.
    dukeandduke, PABrewReview and dcotom like this.
  6. KRug

    KRug Initiate (150) Aug 20, 2018 Massachusetts

    Yeah, the difference between increments of .25 vs quality is very much a parabolic shape in this sense. Above and below 3.5 is where I would say my ratings make a “difference”, for lack of a better word
  7. PapaGoose03

    PapaGoose03 Poo-Bah (2,565) May 30, 2005 Michigan

    He's doing maths! Anything to say about that? :wink:
    beertunes likes this.
  8. beertunes

    beertunes Poo-Bah (6,565) Sep 24, 2007 Saint Martin

    Yeah, I agree with this, but perhaps, for me at least, it's the .5 jumps that are harder. If I'm having two beers that are very similar (both in the say 3.25 range), I'm likely to bump one of the numbers (either up or down) so that they're not given exactly the same score. So, while one beer might end with something in the 3.1 range, and the other nearer to 3.2, to me they're the same beer.

    But, a 3.5 beer is leagues better than the 3.0, and noticeably better than the 3.25. And, no, I am not any of the three things you list at the top of your OP. :wink:
  9. rgordon

    rgordon Meyvn (1,005) Apr 26, 2012 North Carolina

    Some of my very best teachers through the years (there were many) operated this way in dealing with all of the disparities of students and general demographics. Closely watching students that need assistance is the mark of a great teacher. Parcing achievers is not as hard to do and generally works itself out nicely. Rating beers does seem to be a mostly benevolent exercise.
    Marklaker likes this.
  10. TootyMcButtface

    TootyMcButtface Initiate (27) Feb 16, 2019 Kansas

    Why do people withhold 5 star ratings for beer? I know people who refuse to give a beer 5 stars and I have never understood this concept. If I love a beer, I'm happy to give it the highest score possible. And yes, more than one beer is eligible to earn that score.

    To answer the OP, I rate purely by feel. I have arbitrary lines in my mind that go approximately like so:
    >4.5 - one of my favorite beers and something I'd keep in my fridge if possible
    >4 - something I would buy again and I really enjoy
    Between 2.5 and 4 - something I'd happily drink, but wouldn't seek it out or buy it
    Between 1 and 2.5 - basically all your crappy cheap beers
    <1 - something I would not drink even for free.

    The reason I dislike people not using 5 star ratings is because when your theoretical best possible score you're willing to give is 4.25/5, then that's only a B grade to the beer. And that blows my mind. So you've never had a beer that earned an A???

    (Not directed at the OP in particular, I have wondered this for awhile)
  11. beertunes

    beertunes Poo-Bah (6,565) Sep 24, 2007 Saint Martin

    Well, on this website, you're not allowed to give a below 1 score. And, that's quite a jump from 2.5 to 4.

    For me, a 4 is truly great beer, one that gets you excited, and there's relatively few of those. With over 3600 ratings, and over 3200 full reviews, I've only given 4 5s, 126 from 4.25-4.75, and 162 4s. That's 192 beers out of well over 3000 that are that exceptional. On the other han, from 1-2, inclusive, there are only 62 beers. so yes, in my experience there are far more good beers than bad, but the exceptions on either end are very rare.
  12. TootyMcButtface

    TootyMcButtface Initiate (27) Feb 16, 2019 Kansas

    I guess my point is mostly semantics but it's weird that a 4/5 (80%, B- grade) classifies as truly great. In your example, you've rated 292 out of 3000 at 4/5 or better, meaning less than 10% of beers earned a B grade. That just strikes me as odd. You aren't the exception though, I've noticed the general populace tends to grade similarly.

    If beers had to be assigned a score on a scale from 1-100 instead of 1-5, I wonder how different they might look.
    Singlefinpin, KRug and Apellonious like this.
  13. MNAle

    MNAle Poo-Bah (1,657) Sep 6, 2011 Minnesota

    Well, if you consider each increment to be equal to the next on a linear scale, it is worse than that. Since 0 to 1 does not exist, a 4 is 75%.

    If there was an equivalent letter grade, it would definitely be a non-linear curve. Anything above 4.0 would be in the A range (A- to A+).
    rgordon and eppCOS like this.
  14. Mister_Faucher

    Mister_Faucher Defender (620) Dec 3, 2014 Washington

    For the record, to @beertunes in perspective your average beer rating is 3.42, right bro? You admitted that when Todd and crew expanded the ratings options.

    I'll stop short of calling you jaded. But your perspective on quality beers is definitely skewed.

    And we in fact DO have several similar ratings on OUR local beers, on some.
    KRug and beertunes like this.
  15. beertunes

    beertunes Poo-Bah (6,565) Sep 24, 2007 Saint Martin

    Oh, sure. I disagree with the word "skewed" though, I think all the high ratings for hype beers are skewed. We saw it in the other thread, with statements along the lines of "I only buy highly rated beers that I know will be good, so I give them high ratings". In other words folks are rating beers highly because others are rating them highly. It's worse over on UT though. Shrug.

    I've said before that the more you try, the more the more the number moves to average. It can still be a great beer, but it might be one of a hundred great beers of that style I've had, so, it's closer to average. My "average" beer is pretty damn good, but that in and of itself doesn't raise it above average. Shrug.
  16. Mister_Faucher

    Mister_Faucher Defender (620) Dec 3, 2014 Washington

    I'll agree to a degree, there are few and far between beers where I'm like, "Holy Shit!, This is killer!", perfect example over this summer, Pizza Port/Modern Times 'Gentle Reminder' average at best and $16 a four pack!?

    I'd much rather hit Walmart and get a sixer of Cedardust :slight_smile:
    #16 Mister_Faucher, Jan 10, 2020
    Last edited: Jan 10, 2020
    beertunes likes this.
  17. WesMantooth

    WesMantooth Poo-Bah (2,979) Jan 8, 2014 Ohio
    Society Trader

    I’ve never understood the scoring system starting at 1. I wish it would have been that way in high school. It’s like getting credit for putting your name on your paper.
    dukeandduke, rgordon, KRug and 3 others like this.
  18. Griggsy

    Griggsy Zealot (544) Mar 24, 2013 Oregon

    I am definitely a math-minded person, and I definitely am non-linear in my ratings. A beer has to be borderline non-drinkable for me to give below a 3. Typically, 3.5 is what I would consider a perfectly average beer. 3.75 is above average, 4 is good, 4.25 is great, 4.5 is really great, 4.75 is fantastic, and 5 is one of the best beers I've ever had. I know that it makes no sense. based on what a 1 to 5 scale is supposed to be. But I cannot help but rate things in this manner. I don't even record these ratings at this point, but if someone asks me what rating I'd give it, I still end up using this absurdly non-linear scale.
    Singlefinpin and beertunes like this.
  19. Roguer

    Roguer Poo-Bah (4,546) Mar 25, 2013 Georgia
    Moderator Society Trader

    I would go with partially linear, perhaps.

    Meaning: by intent, yes, my ratings are linear. In practice, however, I suspect I'm more like you (@MNAle ), in particular the closer one ranges toward 3.0. I think I'm actually pretty consistent from 3.5 to 5.0, but below 3.5, I'm probably a very generous rater.

    Interesting topic, no doubt.
    dukeandduke, beertunes and Griggsy like this.
  20. Mister_Faucher

    Mister_Faucher Defender (620) Dec 3, 2014 Washington

    So are you saying you would give... say a Goose Island Bourbon Barrel aged a higher rating on a 95 degree day over say... a Gose/Wild on that same day? I tend to take style and the seasonal conditions in to consideration as a determining factor.

    The main factor being my personal experience within the same style.
    beertunes and Roguer like this.
  21. Roguer

    Roguer Poo-Bah (4,546) Mar 25, 2013 Georgia
    Moderator Society Trader

    No, not necessarily. Using stouts as a specific example, I drink stouts year round, regardless of temperature, so I'm not particularly influenced by the season or weather.

    But, your point is valid: while I try to rate on a consistent scale, there are undoubtedly some factors that influence my ratings (whether seasonal or otherwise), at that likely in turn influences just how linear my scores pan out.

    I think, more than anything, the subjective quality of, "How much do I enjoy this?" shifts my overall score toward a particular number, and the more emotional my subjective reaction, the less likely that this influence falls along linear lines. At least, I suspect that is the case. :slight_smile:
  22. Mister_Faucher

    Mister_Faucher Defender (620) Dec 3, 2014 Washington

    I can respect that. Might not be my angle but fair enough. What's your average rating for the record? After updating mine it dropped a bit to be honest.
    Roguer likes this.
  23. Roguer

    Roguer Poo-Bah (4,546) Mar 25, 2013 Georgia
    Moderator Society Trader

    3.97. A little higher than I'd like, although I (perhaps understandably) tended to rate a little high back in my earlier days on the site. Out of my last several hundred ratings, I'd be very surprised if there was a period of 50 reviews where I didn't clock in -rDev on more than half. I tend to view rDev as a more reliable indicator of my ratings than the rating itself, as - like most of us - I tend to buy beer I already suspect I will enjoy, which skews the ratings higher (I don't care how many Bud Light Lime variants they're releasing; I have no intent to review them all :grinning: ).

    I'm mostly just trying to acknowledge my biases, and the tricky thing about biases is: you don't truly know they're there (otherwise, a wise man would strive to mitigate or eliminate them). So to circle back to @MNAle 's original question, my more complete answer would be: I strive for a linear rating methodology, but I suspect I don't actually achieve it in practice. :slight_smile:
    dukeandduke, KRug, beertunes and 2 others like this.
  24. Mister_Faucher

    Mister_Faucher Defender (620) Dec 3, 2014 Washington

    Whereas I cut my teeth on shit beer here and started exploring. First German and Eastern Euro lagers then finally settled in to the IPA realm. My two only 5s are Weihensteph Weiss and Kristal.
  25. Apellonious

    Apellonious Meyvn (1,419) Oct 25, 2008 North Carolina
    Society Trader

    Looking at my beer rating chart, I am in alignment with what @MNAle illustrates in the original post. @TootyMcButtface brings up a good point around lack of fives and questioning how that can happen. In retrospect, I should have more top tier ratings; I have had some great beers, and may have already had my favorite one(s) before this great journey ends.

    To offer up a possible least in my case...initial ratings within a scale acknowledge the fact of the unknown and untried. How I can rate something a 5 when there are "x" beers out there that I haven't tried and are unknown and possibly could be the best? And I have only rated...say...40 beers, at that point in time. Need to build a reference point from experience. And then the beer universe I am interested in expands significantly (true story!!!) due to the macro growth of the overall craft beer movement and numerous other factors. Then there is creation of, exposure to, and my own changing interest in, new styles. More access to local and not local options, palate shift, etc... And with that, the unknown or untried is still out there, even as experience builds. As time goes on you build up a ratings reference point (which may include a number of individual biases; biases is probably not the right word, perhaps the OP can inform or correct me) and that's your starting point for a "good" beer, and you pivot up and down from there. It would be great to have 20/20 hindsight and re-rate your past based on your future experience. Even if the beer still exists to re-rate in the current space, there may be variation with the beer, you, or the experience that makes proper re-rates difficult to quantify against other ratings.

    To provide full disclosure, I had an Old Mecklenburg Captain Jack followed by Lawson's Finest Liquids Fayston Maple Imperial Stout before crafting this message.
    taxandbeerguy, rgordon and beertunes like this.
  26. officerbill

    officerbill Devotee (428) Feb 9, 2019 New York

    In my case it's because I know I'm not drinking “5” level beer. There's no 5 in the top 250 and most on that list are 4.6ish.

    I have six beers rated >4.5 (5 Belgian, 1 German). I consider these to be the best of their styles I'm likely to ever come across or afford.
  27. DEdesings57

    DEdesings57 Zealot (557) Aug 26, 2012 New Jersey

    My issue with numerical beer ratings is that your trying to quantify something that is subjective in nature by means of attaching a scale to it. Of course this doesn't apply only to beer. But it's one of the reasons why I don't take reviews all that seriously.

    But there are fun to get an quick idea of what most people think about a particular beer.

    As for Linearity: It would definitely be a cool idea to create a scatter plot of points on an a Cartesian plane for two variables and see what kind of pattern you see. Hummm I might do this for fun.

    Also as for Linearity I dont think the Standard form of a linear Equation: Ax +By = C gets enough respect and love, y = Mx + B steals all the fun :rofl::joy:
    dukeandduke, PapaGoose03 and MNAle like this.
  28. socon67

    socon67 Poo-Bah (1,867) Jun 18, 2010 New York

    I find with my ratings it is very difficult for me to rate 4.5 or higher
    5 - 1 beer
    4.75-4.99 - 7 beers
    4.50-4.74 - 39 beers

    Out of 2,659 rated beers on my list, that's a small subset. I also see that on the low side as well, with 48 beers rated below a 3.0. I am going to think for most folks on here, there will be their average, with it being exceptional (good or bad) as the outliers.
  29. Brian29

    Brian29 Devotee (468) Nov 15, 2013 Ohio

    I conceive my ratings like school grades. I try hard to liken them to standard deviations on a bell curve.
    I dont have any issue with doling out a 5 in a category. Weve all received a 100% on an assignment. We arent perfect people. But we nailed that task, assignment, or test. A beer to me can certainly nail mouthfeel. Perfect, who knows. Nailed it, yes.

    I reviewed my actual ratings to observe if i were following a bell curve and determined im not due to purchasing habits. After the cutting my teeth period where i tried ev er y thing, i no longer purchase beers i expect, anticipate, or hunch to be a 70% or 3.5.

    So of my reviewed titles, they tend to be 4. to 4.25 as im buying only those i expect to fall into this range. 4.5-5s arent often shelfies. And below 4s im leaving on the shelf or on draught.

    Im most likely to order the pour i expect to land closest to 5 when selecting.
  30. Singlefinpin

    Singlefinpin Aspirant (258) Jul 17, 2018 North Carolina

    I have a hard time giving a beer an overall 4.5, or anything below 3 25.
    Ratings are subjective and some what emotional.
    If, I rate a beer Overall above a 4, it's probably worth trying.
    Below that? No, don't try it
  31. snaotheus

    snaotheus Poo-Bah (3,534) Oct 6, 2008 Washington

    I'm an engineer. I feel you, bro. You've closely described my own ratings, although "it's gotta be pretty awful to go below X" for me, X = 3.

    My own rating system:

    < 3: How bad is it?
    3.0: It's not terrible.
    3.25: It's OK.
    3.5: I'd drink it again but I'd (probably) never seek it out.
    3.75: It's fine. I'd probably recommend it to a fanatic of the style, if it's a style I don't have strong feelings about (if I had strong feelings about it, I'd have stronger recommendations). I wouldn't seek it out. I wouldn't turn it down unless something better or more interesting was available. I would guess by far my most common rating.
    4.0: It's good. Definitely above average. Generally, I'd be happy to get it again, unless it's a style where I can easily get better beers. Or if it's a style I don't care much about, in which case I mostly only seek out beers I haven't tried. Probably noticeably less common than 3.75, but also very common.
    4.25: It's markedly above average. If I gave it a 4.25 and you have any regard for my ratings, this is probably a beer you should try if you get a chance.
    4.5: It's amazing. This is a pretty frickin' large jump from 4.25. Go out of your way to try this beer.
    4.75: My mind is blown. Another large jump.
    5.0: I'm probably drunk, in a great mood, and it's really fucking good. Non-trivial probability that it's a beer I reviewed after several hours at Cantillon.

    Now I'm really dreading the detail oriented person who goes to check my actual ratings against my perceptions of my ratings.
  32. woodchipper

    woodchipper Meyvn (1,143) Oct 25, 2005 Connecticut

    Engineer here. I struggle with ratings a lot because, in the end, its a subjective process. I have not found a good way to make rating a beer an objective, data driven process.
    Having said that, the vast majority of beers I rate are selected based on reputation, so of course my ratings are skewed to high. Yea I've given a very few 5's to some qualities of some beers but never anything close to a 5 across the board.
    With regard to the linearity question, I have very volatile ratings for beers on the low side of my scores, although as I've already stated, they are few and far between due to my selection process.
    In the last two years I have scaled back rating beers significantly. I probably rate 20% of the beers I try, but I do write a review on 95% of those. My cutback is due mostly to trying to be "in the moment" with the beer, the company and the environment that I am drinking in. Taking notes and writing a review seems to just damp some beer experiences.
  33. cjgiant

    cjgiant Poo-Bah (4,943) Jul 13, 2013 District of Columbia

    My ratings are not linear, and after seeing the chart/average with the new feature, I am not surprised. The issue with giving a beer a 5s across the board is that usually there is something lacking in one of the five categories. I haven't run into a beer that has it all going on in one package. My highest rated, BCBS, suffers from look and feel due to the high ABV, for instance.

    I noticed that I rated lower than most on my early reviews as I was reading of all these great beers I hadn't tried yet, I need room for them. Then I realized the great beers I was missing weren't that much better than others. I also realized I was stuck with a glut between 3.5 and 4.25 and that my highest beers at the time (around 4.5) should be closer to 5.

    Whereas I thought I was limiting myself to the high side before, I now feel I am a little too lenient to the high side, especially for local places that make up the bulk of my new reviews/ratings. I am a big fan of re-reviewing/rating with new info, and actually would like to spend more time revisiting ratings and adjusting.

    Note: I also admit to a slight bias for new locals that produce good beer, as I usually expect glorified home brews and get just a little hopeful when the beers are solid. That said, I usually have a negative rDev for local brews - especially for the more common styles.
  34. rgordon

    rgordon Meyvn (1,005) Apr 26, 2012 North Carolina

    Those two sentences remind me of my first encounter with statistics. I was working on a political science, English, and sociology major. Then I ran into Carlo Lastrucci and his Fundamentals of Statistics. It was a required course of study and it was like Cyrillic, unknowable.
  35. puck1225

    puck1225 Poo-Bah (2,499) Aug 14, 2015 Texas

    I am a scientist by training, and I make a good attempt to be objective in my ratings and take into account the style. But, this is for fun, and things like sentimentality, occasion, and othe very subjective factors are sometimes integrated into my ratings.

    For example, just tonight I gave Rolling Rock Extra Pale a higher rating than it probably desereved for old time sake, and I had a great time doing it!
  36. NickSMpls

    NickSMpls Meyvn (1,184) Nov 11, 2012 Washington
    Society Trader

    Engineer here as well. I skimmed the previous posts and it's a lively discussion. Unless I missed it, I didn't see any mention of the fact that we have 5 attributes to rate on and these are heavily weighted as well. From the BA Help section we find
    Appearance 6%
    Smell 24%
    Taste 40%
    Mouthfeel 10%
    Overall 20%

    Appearance is the only attribute that is possibly independent of overall ratings. But it's only 1/16th contribution. A beautiful beer could taste like crap but wouldn't move the needle that much. Poor stays poor. So the higher weighted ratings would tend to cluster together. Smells good = tastes good ( probably).

    Given the sample sizes we are working with here, personal preferences, and the law of large numbers, I'm not surprised we end up with a bell-curve distribution with largish distribution. That means for the typical reviewer, your ratings will bunch up around whatever your median is. As you move up or down from that value, the number of samples (ratings) will of course drop off and so the variations between them will go up.
    Hope that makes sense.
    Marklaker and cjgiant like this.
  37. PapaGoose03

    PapaGoose03 Poo-Bah (2,565) May 30, 2005 Michigan

    I don't do much rating on this site, but I have my own system that I use and log that I keep, and it's also on a 1-5 scoring range. I'll probably give a 5 after I finally give a 1 score to something. I don't want to go outside the classic bell-shaped curve because I purchase most of my beers without any advance info to know what I'm getting. I think a few have reached 4.75 and a few have been at a 2 score, so it's looking like I'm closer to scoring a 5 than I am to a 1.

    But another factor to consider here is that it's probably a lot easier to find a 1 beer than it is to find a 5 beer.
  38. taxandbeerguy

    taxandbeerguy Poo-Bah (2,599) Jul 12, 2013 Ontario (Canada)

    Tax Accountant here with a skew towards statistics and numbers in general. So I relate pretty well with the engineering and math folks. With the updated stats function, I have a giant bell curve centered on my average of 3.69. The beer has to be pretty bad to get scored much less than a 3, I'd cite about 2.8 as that threshold. Similarly, after nearly 2,000 reviews, there's still less than 200 that score 4.25 or better; my top rated beer is 4.84 (Rochefort 10); those are special beers that I'd be willing to enjoy at any time. Generally they make my jaw drop with "wow". Some may not make my jaw drop given the acclaim, but even then they are still phenomenal (Westy 12, Founders KBS and CBS among these-went in with super high expectations and they delivered, but perhaps did not exceed my expectations). I've also only had a handful of beers under a 2. I mean Bud / Coors Light is so mediocre but is it offensive? Do I want to pour it out on the (very) rare occasion I'm drinking it? Generally no.If there's no flavor it's hard to be worse than that.

    And also, probably as many of us have experienced, there was an urgency to try everything in the early going, once, you've been in the game for a few years, now you see a session IPA or Cream Ale (or whatever less favorite style) from a brewery you don't love and haven't had great experiences with; well it's a pass now (as opposed to trying more average-ish-mediocre beer), so the beer I'm reviewing now is probably of a marginally higher quality (to me because I'm actually genuinely interested in trying it), partially from internal bias (drinking less styles that I'm not a fan of) and partially because craft beer has continued to improve, develop and improvise and expand at a great rate in recent years.
    Apellonious and NickSMpls like this.
  39. Marklaker

    Marklaker Zealot (562) Jun 5, 2014 Florida

  40. 57md

    57md Poo-Bah (2,592) Aug 22, 2011 Pennsylvania

    Like you, I am not linear at the top end of the scale. For me, the same hold true for the bottom end of the scale.

    A quick check of my scores indicates that my median score is 3.84 (my mean is 3.81). A majority of my scores are within +/- .2 of my median. So, I have a cluster of scores from the very low 4s through the mid 3.6 range.