Are your beer ratings linear?

Talk Discussion in 'BeerAdvocate Talk' started by MNAle, Jan 9, 2020.

Thread Status:
Not open for further replies.
  1. Roguer

    Roguer Grand High Pooh-Bah (7,811) Mar 25, 2013 Connecticut
    Mod Team Society Pooh-Bah Trader


    No, not necessarily. Using stouts as a specific example, I drink stouts year round, regardless of temperature, so I'm not particularly influenced by the season or weather.

    But, your point is valid: while I try to rate on a consistent scale, there are undoubtedly some factors that influence my ratings (whether seasonal or otherwise), at that likely in turn influences just how linear my scores pan out.

    I think, more than anything, the subjective quality of, "How much do I enjoy this?" shifts my overall score toward a particular number, and the more emotional my subjective reaction, the less likely that this influence falls along linear lines. At least, I suspect that is the case. :slight_smile:
     
  2. Mister_Faucher

    Mister_Faucher Initiate (0) Dec 3, 2014 Washington

    I can respect that. Might not be my angle but fair enough. What's your average rating for the record? After updating mine it dropped a bit to be honest.
     
    Roguer likes this.
  3. Roguer

    Roguer Grand High Pooh-Bah (7,811) Mar 25, 2013 Connecticut
    Mod Team Society Pooh-Bah Trader

    3.97. A little higher than I'd like, although I (perhaps understandably) tended to rate a little high back in my earlier days on the site. Out of my last several hundred ratings, I'd be very surprised if there was a period of 50 reviews where I didn't clock in -rDev on more than half. I tend to view rDev as a more reliable indicator of my ratings than the rating itself, as - like most of us - I tend to buy beer I already suspect I will enjoy, which skews the ratings higher (I don't care how many Bud Light Lime variants they're releasing; I have no intent to review them all :grinning: ).

    I'm mostly just trying to acknowledge my biases, and the tricky thing about biases is: you don't truly know they're there (otherwise, a wise man would strive to mitigate or eliminate them). So to circle back to @MNAle 's original question, my more complete answer would be: I strive for a linear rating methodology, but I suspect I don't actually achieve it in practice. :slight_smile:
     
    ChicagoJ, Rug, BBThunderbolt and 2 others like this.
  4. Mister_Faucher

    Mister_Faucher Initiate (0) Dec 3, 2014 Washington

    Whereas I cut my teeth on shit beer here and started exploring. First German and Eastern Euro lagers then finally settled in to the IPA realm. My two only 5s are Weihensteph Weiss and Kristal.
     
  5. Apellonious

    Apellonious Pooh-Bah (1,814) Oct 25, 2008 North Carolina
    BA4LYFE Society Pooh-Bah Trader

    Looking at my beer rating chart, I am in alignment with what @MNAle illustrates in the original post. @TootyMcButtface brings up a good point around lack of fives and questioning how that can happen. In retrospect, I should have more top tier ratings; I have had some great beers, and may have already had my favorite one(s) before this great journey ends.

    To offer up a possible explanation...at least in my case...initial ratings within a scale acknowledge the fact of the unknown and untried. How I can rate something a 5 when there are "x" beers out there that I haven't tried and are unknown and possibly could be the best? And I have only rated...say...40 beers, at that point in time. Need to build a reference point from experience. And then the beer universe I am interested in expands significantly (true story!!!) due to the macro growth of the overall craft beer movement and numerous other factors. Then there is creation of, exposure to, and my own changing interest in, new styles. More access to local and not local options, palate shift, etc... And with that, the unknown or untried is still out there, even as experience builds. As time goes on you build up a ratings reference point (which may include a number of individual biases; biases is probably not the right word, perhaps the OP can inform or correct me) and that's your starting point for a "good" beer, and you pivot up and down from there. It would be great to have 20/20 hindsight and re-rate your past based on your future experience. Even if the beer still exists to re-rate in the current space, there may be variation with the beer, you, or the experience that makes proper re-rates difficult to quantify against other ratings.

    To provide full disclosure, I had an Old Mecklenburg Captain Jack followed by Lawson's Finest Liquids Fayston Maple Imperial Stout before crafting this message.
     
  6. officerbill

    officerbill Pooh-Bah (2,228) Feb 9, 2019 New York
    Pooh-Bah Trader

    In my case it's because I know I'm not drinking “5” level beer. There's no 5 in the top 250 and most on that list are 4.6ish.

    I have six beers rated >4.5 (5 Belgian, 1 German). I consider these to be the best of their styles I'm likely to ever come across or afford.
     
  7. DEdesings57

    DEdesings57 Pooh-Bah (2,556) Aug 26, 2012 New Jersey
    Pooh-Bah Trader

    My issue with numerical beer ratings is that your trying to quantify something that is subjective in nature by means of attaching a scale to it. Of course this doesn't apply only to beer. But it's one of the reasons why I don't take reviews all that seriously.

    But there are fun to get an quick idea of what most people think about a particular beer.

    As for Linearity: It would definitely be a cool idea to create a scatter plot of points on an a Cartesian plane for two variables and see what kind of pattern you see. Hummm I might do this for fun.

    Also as for Linearity I dont think the Standard form of a linear Equation: Ax +By = C gets enough respect and love, y = Mx + B steals all the fun :rofl::joy:
     
    ChicagoJ, PapaGoose03 and MNAle like this.
  8. socon67

    socon67 Grand Pooh-Bah (3,895) Jun 18, 2010 New York
    Pooh-Bah

    I find with my ratings it is very difficult for me to rate 4.5 or higher
    5 - 1 beer
    4.75-4.99 - 7 beers
    4.50-4.74 - 39 beers

    Out of 2,659 rated beers on my list, that's a small subset. I also see that on the low side as well, with 48 beers rated below a 3.0. I am going to think for most folks on here, there will be their average, with it being exceptional (good or bad) as the outliers.
     
  9. Brian29

    Brian29 Initiate (0) Nov 15, 2013 Ohio
    Trader

    I conceive my ratings like school grades. I try hard to liken them to standard deviations on a bell curve.
    I dont have any issue with doling out a 5 in a category. Weve all received a 100% on an assignment. We arent perfect people. But we nailed that task, assignment, or test. A beer to me can certainly nail mouthfeel. Perfect, who knows. Nailed it, yes.

    I reviewed my actual ratings to observe if i were following a bell curve and determined im not due to purchasing habits. After the cutting my teeth period where i tried ev er y thing, i no longer purchase beers i expect, anticipate, or hunch to be a 70% or 3.5.

    So of my reviewed titles, they tend to be 4. to 4.25 as im buying only those i expect to fall into this range. 4.5-5s arent often shelfies. And below 4s im leaving on the shelf or on draught.

    Im most likely to order the pour i expect to land closest to 5 when selecting.
     
  10. Singlefinpin

    Singlefinpin Pooh-Bah (2,400) Jul 17, 2018 North Carolina
    Pooh-Bah Trader

    I have a hard time giving a beer an overall 4.5, or anything below 3 25.
    Ratings are subjective and some what emotional.
    If, I rate a beer Overall above a 4, it's probably worth trying.
    Below that? No, don't try it
     
  11. snaotheus

    snaotheus Grand High Pooh-Bah (7,924) Oct 6, 2008 Washington
    BA4LYFE Society Pooh-Bah Trader

    I'm an engineer. I feel you, bro. You've closely described my own ratings, although "it's gotta be pretty awful to go below X" for me, X = 3.

    My own rating system:

    < 3: How bad is it?
    3.0: It's not terrible.
    3.25: It's OK.
    3.5: I'd drink it again but I'd (probably) never seek it out.
    3.75: It's fine. I'd probably recommend it to a fanatic of the style, if it's a style I don't have strong feelings about (if I had strong feelings about it, I'd have stronger recommendations). I wouldn't seek it out. I wouldn't turn it down unless something better or more interesting was available. I would guess by far my most common rating.
    4.0: It's good. Definitely above average. Generally, I'd be happy to get it again, unless it's a style where I can easily get better beers. Or if it's a style I don't care much about, in which case I mostly only seek out beers I haven't tried. Probably noticeably less common than 3.75, but also very common.
    4.25: It's markedly above average. If I gave it a 4.25 and you have any regard for my ratings, this is probably a beer you should try if you get a chance.
    4.5: It's amazing. This is a pretty frickin' large jump from 4.25. Go out of your way to try this beer.
    4.75: My mind is blown. Another large jump.
    5.0: I'm probably drunk, in a great mood, and it's really fucking good. Non-trivial probability that it's a beer I reviewed after several hours at Cantillon.

    Now I'm really dreading the detail oriented person who goes to check my actual ratings against my perceptions of my ratings.
     
  12. woodchipper

    woodchipper Grand Pooh-Bah (3,735) Oct 25, 2005 Connecticut
    Society Pooh-Bah Trader

    Engineer here. I struggle with ratings a lot because, in the end, its a subjective process. I have not found a good way to make rating a beer an objective, data driven process.
    Having said that, the vast majority of beers I rate are selected based on reputation, so of course my ratings are skewed to high. Yea I've given a very few 5's to some qualities of some beers but never anything close to a 5 across the board.
    With regard to the linearity question, I have very volatile ratings for beers on the low side of my scores, although as I've already stated, they are few and far between due to my selection process.
    In the last two years I have scaled back rating beers significantly. I probably rate 20% of the beers I try, but I do write a review on 95% of those. My cutback is due mostly to trying to be "in the moment" with the beer, the company and the environment that I am drinking in. Taking notes and writing a review seems to just damp some beer experiences.
     
  13. cjgiant

    cjgiant Grand High Pooh-Bah (6,584) Jul 13, 2013 District of Columbia
    Society Pooh-Bah

    My ratings are not linear, and after seeing the chart/average with the new feature, I am not surprised. The issue with giving a beer a 5s across the board is that usually there is something lacking in one of the five categories. I haven't run into a beer that has it all going on in one package. My highest rated, BCBS, suffers from look and feel due to the high ABV, for instance.

    I noticed that I rated lower than most on my early reviews as I was reading of all these great beers I hadn't tried yet, I need room for them. Then I realized the great beers I was missing weren't that much better than others. I also realized I was stuck with a glut between 3.5 and 4.25 and that my highest beers at the time (around 4.5) should be closer to 5.

    Whereas I thought I was limiting myself to the high side before, I now feel I am a little too lenient to the high side, especially for local places that make up the bulk of my new reviews/ratings. I am a big fan of re-reviewing/rating with new info, and actually would like to spend more time revisiting ratings and adjusting.

    Note: I also admit to a slight bias for new locals that produce good beer, as I usually expect glorified home brews and get just a little hopeful when the beers are solid. That said, I usually have a negative rDev for local brews - especially for the more common styles.
     
  14. rgordon

    rgordon Pooh-Bah (2,701) Apr 26, 2012 North Carolina
    Pooh-Bah

    Those two sentences remind me of my first encounter with statistics. I was working on a political science, English, and sociology major. Then I ran into Carlo Lastrucci and his Fundamentals of Statistics. It was a required course of study and it was like Cyrillic, unknowable.
     
  15. puck1225

    puck1225 Grand Pooh-Bah (4,585) Dec 22, 2013 Texas
    Pooh-Bah Trader

    I am a scientist by training, and I make a good attempt to be objective in my ratings and take into account the style. But, this is for fun, and things like sentimentality, occasion, and othe very subjective factors are sometimes integrated into my ratings.

    For example, just tonight I gave Rolling Rock Extra Pale a higher rating than it probably desereved for old time sake, and I had a great time doing it!
     
  16. NickSMpls

    NickSMpls Grand Pooh-Bah (3,176) Nov 11, 2012 Washington
    Society Pooh-Bah Trader

    Engineer here as well. I skimmed the previous posts and it's a lively discussion. Unless I missed it, I didn't see any mention of the fact that we have 5 attributes to rate on and these are heavily weighted as well. From the BA Help section we find
    Appearance 6%
    Smell 24%
    Taste 40%
    Mouthfeel 10%
    Overall 20%

    Appearance is the only attribute that is possibly independent of overall ratings. But it's only 1/16th contribution. A beautiful beer could taste like crap but wouldn't move the needle that much. Poor stays poor. So the higher weighted ratings would tend to cluster together. Smells good = tastes good ( probably).

    Given the sample sizes we are working with here, personal preferences, and the law of large numbers, I'm not surprised we end up with a bell-curve distribution with largish distribution. That means for the typical reviewer, your ratings will bunch up around whatever your median is. As you move up or down from that value, the number of samples (ratings) will of course drop off and so the variations between them will go up.
    Hope that makes sense.
     
    Marklaker and cjgiant like this.
  17. PapaGoose03

    PapaGoose03 Grand High Pooh-Bah (6,057) May 30, 2005 Michigan
    BA4LYFE Society Pooh-Bah

    I don't do much rating on this site, but I have my own system that I use and log that I keep, and it's also on a 1-5 scoring range. I'll probably give a 5 after I finally give a 1 score to something. I don't want to go outside the classic bell-shaped curve because I purchase most of my beers without any advance info to know what I'm getting. I think a few have reached 4.75 and a few have been at a 2 score, so it's looking like I'm closer to scoring a 5 than I am to a 1.

    But another factor to consider here is that it's probably a lot easier to find a 1 beer than it is to find a 5 beer.
     
  18. taxandbeerguy

    taxandbeerguy Pooh-Bah (2,799) Jul 12, 2013 Canada (ON)
    Pooh-Bah

    Tax Accountant here with a skew towards statistics and numbers in general. So I relate pretty well with the engineering and math folks. With the updated stats function, I have a giant bell curve centered on my average of 3.69. The beer has to be pretty bad to get scored much less than a 3, I'd cite about 2.8 as that threshold. Similarly, after nearly 2,000 reviews, there's still less than 200 that score 4.25 or better; my top rated beer is 4.84 (Rochefort 10); those are special beers that I'd be willing to enjoy at any time. Generally they make my jaw drop with "wow". Some may not make my jaw drop given the acclaim, but even then they are still phenomenal (Westy 12, Founders KBS and CBS among these-went in with super high expectations and they delivered, but perhaps did not exceed my expectations). I've also only had a handful of beers under a 2. I mean Bud / Coors Light is so mediocre but is it offensive? Do I want to pour it out on the (very) rare occasion I'm drinking it? Generally no.If there's no flavor it's hard to be worse than that.

    And also, probably as many of us have experienced, there was an urgency to try everything in the early going, once, you've been in the game for a few years, now you see a session IPA or Cream Ale (or whatever less favorite style) from a brewery you don't love and haven't had great experiences with; well it's a pass now (as opposed to trying more average-ish-mediocre beer), so the beer I'm reviewing now is probably of a marginally higher quality (to me because I'm actually genuinely interested in trying it), partially from internal bias (drinking less styles that I'm not a fan of) and partially because craft beer has continued to improve, develop and improvise and expand at a great rate in recent years.
     
    Apellonious and NickSMpls like this.
  19. Marklaker

    Marklaker Pooh-Bah (2,099) Jun 5, 2014 Florida
    Pooh-Bah

    :+1:
     
  20. 57md

    57md Grand Pooh-Bah (3,033) Aug 22, 2011 Pennsylvania
    Pooh-Bah

    Like you, I am not linear at the top end of the scale. For me, the same hold true for the bottom end of the scale.

    A quick check of my scores indicates that my median score is 3.84 (my mean is 3.81). A majority of my scores are within +/- .2 of my median. So, I have a cluster of scores from the very low 4s through the mid 3.6 range.
     
Thread Status:
Not open for further replies.