How Have Your Ratings Changed Over Time?

Discussion in 'Beer Talk' started by Roguer, Nov 26, 2014.

Thread Status:
Not open for further replies.
  1. Roguer

    Roguer Grand High Pooh-Bah (7,811) Mar 25, 2013 Connecticut
    Mod Team Society Pooh-Bah Trader

    ...or have they? I've noticed a significant trend in my own ratings. Especially being an active participant in the New Beer Sunday thread, I notice when I get a string of -rDevs (or, rarely now, a string of +rDevs).

    I ran the numbers to confirm what I already thought. If I remove 0.0% rDevs from the list, and look at +rDevs as a percentage of the remaining reviews (the vast majority, obviously), it breaks down like this:

    57.8% +rDev ratings for the first 175 reviews
    53.9% +rDev ratings for the first 325 reviews (just shy of the halfway mark)
    39% +rDev ratings for the remaining 400 reviews

    So, you can see there's a general downward trend overall, crossing over from 50/50 +/- right around review #225. Moreover, there's a significant downward trend after 325 reviews.

    In plain language: I am far, far more likely to give a beer a rating that is below average now than I was when I started out - even when I had a couple hundred reviews under my belt.

    So, the question is: have you noticed a similar - or opposite - trend in yourself? As you have become a more experienced beer drinker (and reviewer), have you tended to become more critical, or more accepting? Or have your ratings not really changed at all over time?

    I'm curious if others have followed the same basic downward trend. I would expect it's pretty common, but I'm not running the numbers for anyone else. :wink:
     
  2. TongoRad

    TongoRad Grand Pooh-Bah (3,884) Jun 3, 2004 New Jersey
    Society Pooh-Bah Trader

    New Beer Sunday has helped clarify how I assign ratings, myself. I used to just rely on the 'Pull Down' choices, but since they have changed in meaning a number of times, the older review numbers don't seem to mean the same thing these days. For example- if a 'Taste' of 4 was assigned as 'Good', and I thought it was a little better than good, I'd give it a 4.5. (I don't even think there were .25 increments at that point, but don't recall exactly).

    Now I'm just going with my own (hopefully) consistent assignment. As are the 4s (A- is 4.3 or less, and A+ is 4.7 or more), Bs are the 3s, Cs are the 2s and Ds are the 1s. So far it's worked out, even when breaking down attributes.

    In my own head, though, I still take style and context into account. As to trends, it's impossible for me to say due to all of the changes in the system.
     
  3. travolta

    travolta Maven (1,369) Feb 4, 2004 California
    Society

    This is exactly why I do not like it when BA makes changes to reviewing standards. I love to go back and read my reviews and look at the dates and rDev's.
    In response to your post, I think my palate has narrowed a bit over time. I know what i like and tend to buy that. Even if I haven't had the beer before, I know which styles or breweries I will prefer and my rating (I know it shouldn't) will reflect that. It's difficult. As an example: I am not a fan of Belgians. Is it no surprise that Belgians have a lower rDev on my ratings? However, that is a lot of the fun in reading other BA's reviews.
     
    Andrew041180 and Jugs_McGhee like this.
  4. travolta

    travolta Maven (1,369) Feb 4, 2004 California
    Society

    It is not an easy task to take style into account when chili, sriracha, or some other wild ingredient is involved. What's next, worcester sauce in beer?
     
    TongoRad likes this.
  5. TongoRad

    TongoRad Grand Pooh-Bah (3,884) Jun 3, 2004 New Jersey
    Society Pooh-Bah Trader

    That's why I also use 'context'- is it a sipper, a quaffer, a change of pace, etc.

    Just don't call me when the anchovies and tamarind beers come around, though. Well....maybe just the tamarind... :wink:
     
    RobinLee and herrburgess like this.
  6. BBThunderbolt

    BBThunderbolt Grand High Pooh-Bah (7,846) Sep 24, 2007 Kiribati
    Pooh-Bah Trader

    My overall rating has dropped with the more beer I drink. While there tons of great beers out there, in all honesty, there's not much difference between them. If I try 10 new IPAs, they probably are going to have a hard time being notably better than the hundreds I've already had, so their scores will likely be in the low-to-mid 3's.

    My personal guidelines:
    5s- I can't see this beer being any better for its style
    4.5- A rare level of greatness
    4- Excellent beer, would drink it anytime
    3.5- Way better than average for its style
    3-Solid, nicely drinkable, pretty typical for its style
    2.5-Not truly "bad", but needs some improvement
    2- Lots of help needed.
    Below 2- Terrible, no redeeming aspects.
     
    #6 BBThunderbolt, Nov 26, 2014
    Last edited: Nov 26, 2014
    TheDoctor, cavedave, crob3888 and 8 others like this.
  7. THANAT0PSIS

    THANAT0PSIS Pooh-Bah (2,275) Aug 3, 2010 Wisconsin
    Pooh-Bah

    I am in a downtrend as well. I think it's the result of having a greater contextual lens to look at beer through. I rate about 65-35% in favor of style to enjoyment, so the more beers I have from a given style, the better I can understand and critique the next beer I rate in a style. Early on, I had no context, so I would rate everything a lot higher than it really should be because I had nothing to compare it to. My palate has become more refined as well. Nowadays, it's different.
     
    TheDoctor, Roguer and BBThunderbolt like this.
  8. Tilley4

    Tilley4 Pooh-Bah (2,811) Nov 13, 2007 Tennessee
    Society Pooh-Bah Trader

    I am definitely more friendly to Belgians nowadays than I was previously.
     
  9. are_doubleyou

    are_doubleyou Initiate (0) Aug 3, 2014 Illinois

    I honestly lurked on this site for a couple years before I created my account and started rating because I thought I would overrate beers from inexperience. I think it was a good decision to let my palate catch up with my zeal because I can think of a couple dozen beers that I used to love, but now only like because I've had better examples of the style. I might adjust some scores in the future, but I've got about 1,000 different beers and homebrewing experience under my belt now and I know what I want from different styles so I think the trend will only be narrowly downwards moving forward.

    I pretty much rate:
    5: Perfection and I never give this for every attribute.
    >4.5: One of the best of its style for me and something I will seek out.
    >4: A great example of the style and something I will drink regularly and recommend to everyone.
    >3.5: A very good example of the style and something I will gladly drink with reserved praise to others.
    >3: A good example of the style and something I will shrug and drink if no other options present themselves.
    >2.5: A substandard example of the style, but not bad enough that I won't finish a six pack if I took a risk.
    >2: A poor example of the style and if I have multiple bottles I will try to find someone who likes it. Usually a macro drinker who doesn't care.
    <2: I hope I never experience this. I wouldn't even give it away if it was that bad.

    I don't foresee anything moving between these once I have had a pint.
     
    #9 are_doubleyou, Nov 26, 2014
    Last edited: Nov 26, 2014
    DoubleSimcoe likes this.
  10. Fox82791

    Fox82791 Initiate (0) Jun 20, 2014 New York

    My ratings now are lower than when I started because I have had some amazing beers and am more demanding of the beers I drink.
     
    russpowell and MartzMimic like this.
  11. DoubleSimcoe

    DoubleSimcoe Pooh-Bah (2,779) Jan 7, 2009 Pennsylvania
    Pooh-Bah

    I applaud you. More people should be like you. The amount of BA folks who freely give out "All 5's" to beers and places is truly appalling.
     
    david18 and TheDoctor like this.
  12. russpowell

    russpowell Grand High Pooh-Bah (8,292) May 24, 2005 Arkansas
    BA4LYFE Society Pooh-Bah Trader

    LikeSteveD, over time I think my bar is a little higher for the most part. One of things that really plays into that is the lack of faithfulness to the style is so pervasive that it usually has me in a negative frame of mind even if I like what that did from a pure enjoyment standpoint. Hell maybe even more styles are necessary...
     
  13. russpowell

    russpowell Grand High Pooh-Bah (8,292) May 24, 2005 Arkansas
    BA4LYFE Society Pooh-Bah Trader

    Turn that around & look at the ratings of certain styles, there are no "world class" examples of more than a few styles, that is just insane..
     
    TongoRad likes this.
  14. PittBeerGirl

    PittBeerGirl Pooh-Bah (2,423) Feb 27, 2007 Ohio
    Pooh-Bah

    interesting topic.

    I've noticed that when the rdev subject comes up a lot of my extreme reviews are from when I started reviewing.

    24 of my top 50 +rdev reviews are from 2007 (8 of top 10)
    16/50 of my bottom -rdev are from 2007 (still 32%)
    so 40% of extreme reviews from 2007 (about 20% of total reviews from 2007)
     
    Roguer likes this.
  15. JasW

    JasW Pundit (897) Jul 11, 2014 Florida

    I find it interesting to look at which of my rdevs are +/- more than 10%. I have a lot more minuses in that category than pluses.
     
  16. Wolfhead

    Wolfhead Pundit (795) Sep 1, 2009 Illinois

    Sex the first few hundred times was all 4's and 5's and the beer sucked, now? Let's say there's more 2's and 3's but the beer is better
     
  17. Andrew041180

    Andrew041180 Initiate (0) Mar 15, 2013 Massachusetts

    Someone else said that his palate has narrowed after trying lots of beers. I think the same is likely happening to me. I'm getting better at determining what I like and this is trend is finding its ways into my ratings. On the other hand, when I have determined what I like, certain beers go up against the average. For instance, I originally rated Allagash White highly compared to its BA average. I haven't re-rated it, but I now consider it to be very close to a five. I wouldn't change a thing about it.
     
    Roguer likes this.
  18. socon67

    socon67 Grand Pooh-Bah (3,895) Jun 18, 2010 New York
    Pooh-Bah

    I generally rate towards the mean. Don't trash any beers, don't find many that are the best in style I've ever had. Over time I think (looking quickly at my rDev%) that I'm less hype driven and more bullish on style. If its true to what I expect in the beer I'll rate it well.
     
  19. Andrew041180

    Andrew041180 Initiate (0) Mar 15, 2013 Massachusetts

    Hypocrite. You were one of those people once. Looks like between November of 2011 and February of 2012. Nice 20+% positive RDevs for 5's. This is exactly the point of the thread. People's tastes and ratings change - there is no reason to bash people for rating a beer as a 5 if the beer is special to that reviewer.

    I have some 5's myself that if I drank them again would probably come down. Same with 4.5's. However, at the time I hadn't drank as much and they were special beers to me. Dreadnaught from FFF for example. It was once one of the highest rated beers on this site and to me it was special (I had it the month before I joined this site - so I hadn't heard about it when I bought it). I'd love to try it again, but who knows if that will ever happen.
     
  20. yemenmocha

    yemenmocha Grand Pooh-Bah (4,116) Jun 18, 2002 Arizona
    Pooh-Bah

    They've changed the scoring in a variety of ways over the years so I'm not sure how fruitful a longterm look at it would be. Drinkability is gone. There was an overall category. Then for awhile there were no categories. The verbal descriptions changed. I could be wrong, but we might have even had alphabet letter grades too at one point.
     
    BBThunderbolt likes this.
Thread Status:
Not open for further replies.