New Rating System Feedback

Talk Discussion in 'BeerAdvocate Talk' started by sd123, Dec 18, 2014.

Thread Status:
Not open for further replies.
  1. sd123

    sd123 Pooh-Bah (2,073) Jan 15, 2012 Georgia
    Pooh-Bah

    Maybe Todd could shed some light on the reason behind the changes?

    I think it comes down to the fact that people use the site in different ways, some write full reviews others just enter numbers for attributes while others enter one number as a single overall score. The strength of the site however, comes from the averaging of all those ratings. This new system favors one type of user over another and I think it will ultimately have a negative effect on all.
     
    iTunesUpdates, Hrodebert and anteater like this.
  2. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

  3. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    This had nothing to do with our reasons behind the update.
     
  4. anteater

    anteater Pooh-Bah (1,936) Sep 10, 2012 Oregon
    Pooh-Bah Trader

    You're points are all valid and I won't argue with any of them. I suppose a lot of it ultimately just boils down to preference and ideology. Bottom line for me is that I trust myself to come up with a thoughtful score on my own without the help of an algorithm. If I want to write a review, I will write a review. My category scores may not reflect how I actually felt about that category though. I will tweak them to get the score that represents how I feel about the beer.

    As for this, you're both right. I didn't realize that was explicitly stated and will adjust my rating habits accordingly.
     
    navband and SoCalBeerIdiot like this.
  5. SoCalBeerIdiot

    SoCalBeerIdiot Pooh-Bah (2,191) Mar 10, 2013 California
    Pooh-Bah Trader

    So many rules! I rate for me, not for you so don't tell me (or others) how we should or shouldn't be rating.
     
    Hrodebert and sd123 like this.
  6. SoCalBeerIdiot

    SoCalBeerIdiot Pooh-Bah (2,191) Mar 10, 2013 California
    Pooh-Bah Trader

    I understand there is a "proper" way to review beers but I feel it's unrealistic to expect all 400K+ users (or whatever it is now) to follow the exact same method as everyone else. One of the beauties of the last iteration of the rating system was that it was flexible--if you wanted to sit down and write a full-blown review using all the attributes, no problem; if you wanted to update your ratings off of notes once a month and do it quickly (like I usually do), quick, single ratings made that possible.

    Now by forcing (yes, forcing) me to use the attributes model, all my old ratings (over 1100 now) won't match up with any new ratings I enter. Apples and oranges. Apples being my old ratings (where 4 = drink again, 4.5 meant stand in line to drink it again, and a 5 meant I'd drive 1-3 hours AND stand in line to drink it again) and oranges being the new ratings under the attributes model which will now have to be manipulated if I wanted them to come anywhere near my old system. I'm not sure what I'm going to do yet, to be honest. All I know is I'm more than a little bummed by the new rigidity of this system.

    What about having a check box (or separate score altogether) showing a beer's score from written reviews or using all attributes and a different score showing quick ratings? That would leave flexibility in the system (esp. for people who score a lot of beers at once) and yet allow people who take beers' scores super seriously to get what they want, too?
     
  7. kevanb

    kevanb Pooh-Bah (2,705) Apr 4, 2011 Illinois
    Pooh-Bah Trader

    If you rate for yourself, you should be using a different service, like Untappd, where aggregate ratings aren't really significant and the more important user experience are the individualized reviews. What makes the reviews and scores on this site different is that the aggregate reviews play such a crucial role to the community. People use the database here for reference either when at tastings, when shopping, when discussing and/or researching new beers that might be of interest to them. Having a beer say 4/5 does nothing for a user trying to determine if, for example, a saison is too funky and yeasty for them or if it's going to be a bit more approachable to a novice drinker, the written information is absolutely critical.

    If you are hung up on this new method, you could easily just rate everything a 4 across the board, or a 4.5 or a 5 across the board, though I sincerely hope that taking a look at the various attributes encourages you think a bit more critically about what you are drinking. That being said, obviously there isn't a review police and no one can tell you how to review beers, people here have different scales and curves that influence their scoring system and while those vary from person to person, there is generally a common significant trend overall, which is what is important.
     
    BBThunderbolt likes this.
  8. SoCalBeerIdiot

    SoCalBeerIdiot Pooh-Bah (2,191) Mar 10, 2013 California
    Pooh-Bah Trader

    I guess what I'm trying to say is, if everyone on here is expected to take ratings THAT seriously, I think fewer and fewer people will bother to rate on here. I mean, how many current, regular raters write full reviews or even use all the attributes? My guess is it's less than 50%. If the goal is to alienate everyone else, then this seems the way to go. That's far more likely a scenario than believing everyone will all the sudden start taking this stuff so much more seriously that they'll take the time to use this new system the way it was meant to be used and start writing out useful or thoughtful reviews.

    My suggestion of having two different scores would keep both parties engaged and give them both what they want. Just sayin'.
     
    JLaw55 and Hrodebert like this.
  9. Bitterbill

    Bitterbill Grand High Pooh-Bah (7,036) Sep 14, 2002 Wyoming
    BA4LYFE Society Pooh-Bah

    I love that it's back to very near what it once was. The only minor quibble from me is that the Drinkability category is something I miss. Put that back and give it a rating and whatever the final rating is, ends up being the Overall rating. I reckon it would take away some of the guess work, read be more accurate, for what you really think aboot the beer.
     
  10. Rekrule

    Rekrule Initiate (0) Nov 11, 2011 Massachusetts

    How can you call review text optional but have a 5 word minimum? Words have meanings. A forced text minimum means the review text is anything but optional. Luckily 'I do not write reviews' is 5 words. This is the least friendly system to the casual rater among the available options. Also needing to write a full review if I don't like something most people love or vice versa is silly. It either encourages dishonesty or not rating a beer when your opinion varies greatly from the hivemind.
     
    iTunesUpdates and JLaw55 like this.
  11. sd123

    sd123 Pooh-Bah (2,073) Jan 15, 2012 Georgia
    Pooh-Bah

    Agreed. There will always be people who write out full reviews and those who are interested can read them but for everyone else a simple number will do just fine
     
    JLaw55 likes this.
  12. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    It's not arbitrary. Our weighting is inspired by BJCP scoring, which also puts more weight into taste and smell.
     
  13. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Because writing a written review is optional. Unless of course your rating is an outlier. If this occurs you'll be prompted to explain to everyone why your rating is extraordinarily different when compared to the average. See the announcement.
     
    Bitterbill likes this.
  14. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Based on user feedback and confusion over the term, Drinkability was renamed to Overall ages ago as it was found that the vast majority of users were using it as such. And on a side-note: Overall is also a BJCP attribute, of which drinkability is a part of too.
     
  15. Rekrule

    Rekrule Initiate (0) Nov 11, 2011 Massachusetts

    If I HAVE to write 5 words the review isn't optional. Something tells me if my 5 words are blah blah blah blah blah my review will be deleted. 5 mandatory words makes the contents of people who just want to rate's reviews look silly. That furthers the myth that it is somehow worth less or more meaningless than someone who chooses to write a full review. It's already a hostile environment for raters, this just pushes that forward. Somehow this is still friendly to the casual rater?
     
  16. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania
    In Memoriam

    Its just fine for the casual rater because one does not have to write any words at all to rate a beer.
     
    #36 drtth, Dec 19, 2014
    Last edited: Dec 19, 2014
  17. Rekrule

    Rekrule Initiate (0) Nov 11, 2011 Massachusetts

    Odd because I was just forced to minutes ago. The rules state I must write 5 words.
     
  18. drtth

    drtth Initiate (0) Nov 25, 2007 Pennsylvania
    In Memoriam

    Well I just did it (a rating) without any review at all. (See the most recent beer added to my list.) It seems the five word minimum is if you chose to both rate and review.
     
  19. SHODriver

    SHODriver Pooh-Bah (2,415) Aug 13, 2010 North Carolina
    Pooh-Bah

    unless your review is an 'outlier' which is outside of the experiences of the other people.

    I've always been a fan of reviewing vs rating so this makes it feel easier. My biggest complaint with the previous rate/review system is that I would have to score it and then review it. I'd try and score it based upon the average review and then slide to the left or right as I saw fit.

    FWIW I always thought drink-ability was kind of subjective because some high ABV beers are amazing but drinking a lot of it is more than a small challenge. Overall is much clearer
     
  20. Rekrule

    Rekrule Initiate (0) Nov 11, 2011 Massachusetts

    I must have hit an outlier already then with a review of average. Ridiculous. Homers and style boosters are forcig me to write about a meh beer. Awesome.
     
Thread Status:
Not open for further replies.