New Rating System Feedback

Talk Discussion in 'BeerAdvocate Talk' started by sd123, Dec 18, 2014.

Thread Status:
Not open for further replies.
  1. sd123

    sd123 Pooh-Bah (2,073) Jan 15, 2012 Georgia
    Pooh-Bah

    Wasnt sure where to put this post but I was wondering what people think of the new rating system. I know people use the site in different ways but I for one find the new system to be a real step backwards. For years I've been using beeradvocate to rate beers with a single score so I could keep track of which beers I liked and which I didnt and to see what others thought of the same beer. I found the addition of precise ratings to be a real improvement as I was able to assign a beer a 3.4 vs. 3.6 instead of just being able to assign it a 3.5.

    The new system however means that I am forced to assign a number to each attribute just to give a beer a single overall score which is annoying especially since after years of drinking good beer I still cant imagine what the difference is between beers that get a 1 vs. a 5 in the looks department.

    Anyway I was just wondering what other people thought about the new system and how it works with how you use the site.

    Thanks
     
  2. mikeburd1128

    mikeburd1128 Maven (1,409) Oct 28, 2011 New Jersey

    I wouldn't get too worried about it. The ratings system will most likely change again in 6 months anyway.

    EDIT: But I guess to give you a real answer, I do think it's kind of annoying. It's much easier for me to use an app like untappd while I'm drinking and then come back once a month and input all my ratings onto here. I think I have enough experience at this point to know where I'm rating and why I'm rating without having to do the attributes. It seems to me that these forced attributes are a result of too many 5's being thrown around or something. I dk.
     
    pmarlowe, yemenmocha and anteater like this.
  3. Stinky-Dinkins

    Stinky-Dinkins Devotee (389) Feb 5, 2013 Wisconsin

    I give the new rating system a 7.5 / G
     
  4. JLaw55

    JLaw55 Pooh-Bah (2,417) Jul 10, 2014 Missouri
    Society Pooh-Bah

    I am in a bit of the same boat, I like it in some ways, but not in others. I only ever used the quarter intervals for my ratings anyways (3.25, 3.5, 3.75, 4, etc.), so the removal of exact scores does not bother me. I don't like the fact though that I want to put a 3 on a beer that I find just okay, yet it gets a 4 because while the flavor was only okay, I put that it smelled nice and the final score went up. I use the ratings in the same way that I think the OP does, as a way of rating what I like and don't like so I can reference it later. I do not rate based on how good the beer is in reflection to it's style, I rate by if I liked that beer or not.

    Also, I understand trying to detect outliers on scoring, but now when I find a beer that I didn't like or one I really loved, I am forced to write a review? I don't care for doing many reviews, because it becomes work to me. I am trying to enjoy the beer, not analyze and write down notes on it's carbonation content and head retention.
     
  5. clayrock81

    clayrock81 Grand Pooh-Bah (4,621) Nov 6, 2007 Florida
    Pooh-Bah

    I like the scoring system but prefer the other system of "precise" scoring better.
    That being said, I've always thought the percentages were a little offskew and that appearance and smell should lose percentage points and taste have more of a rating.
    As already posted by some, a mediocre to just alright tasting beer should not score as close or equal to a beer that looks/smells blah but tastes great.
    I know there are some chefs/brewers about to remind me that you eat with your eyes/sense of smell is greater than taste but bottom line, I'll take taste over appearance/aroma any time.
     
  6. geocool

    geocool Savant (1,233) Jun 21, 2006 Massachusetts

    I want to just enter the rating in quarter point increments and be done with it. More detailed ratings should be for those who choose to write a review.
     
  7. kevanb

    kevanb Pooh-Bah (2,705) Apr 4, 2011 Illinois
    Pooh-Bah Trader

    This thread should really be called, The "new" old rating system, and yes I love it, I understand the desire to want to just enter one arbitrary number and be done with it, but actually putting some thought into it is so much more useful to the members who read reviews or breweries looking for feedback. I'll admit and I say I hated the quick reviews because I felt people were way too generous and too quick to give something good a 5 and all it did was encourage ticking rather than encourage honest reviews.

    I am very happy.
     
    F2brewers, Todd, TMoney2591 and 6 others like this.
  8. anteater

    anteater Pooh-Bah (1,936) Sep 10, 2012 Oregon
    Pooh-Bah Trader

    Several reasons I'm very disappointed by the removal of the precise scoring and single scoring in .25 intervals:
    • I don't weigh look, smell, taste, mouthfeel and overall the same as BA.
    • Myself and others (including at least one BA moderator) will just end up tweaking certain category's scores to end up with the final score we desire, thus the individual category scores don't reflect how we felt about the beer.
    • It takes too long: I've written 76 full reviews and will continue to do so when the beer is exceptional. If I'm at a bottle share and want to rate 30 new beers, it will be a huge inconvenience.
    • No new functionality, they simply removed one aspect of a very flexible system.
    I pride myself on using BA instead of Untappd because the ratings actually mean something on this site, I'd hate to see the new scoring system drive people away.
     
  9. JLaw55

    JLaw55 Pooh-Bah (2,417) Jul 10, 2014 Missouri
    Society Pooh-Bah

    Just a suggestion, but I think one very easy fix for the scoring issue, would be to still keep it the way it is, but have the overall score that you give, be the score that shows up for your rating. Not taking a mathematical number from the scores you gave to taste, smell, look, feel, overall. This way you could still break down your scoring of that specific beer, but still be able to mark an okay beer with a 3, instead of it being marked with a 3.87 because you gave it the redeeming quality of look or smell.
     
    ExVashonGujy likes this.
  10. twb0392

    twb0392 Initiate (0) Aug 1, 2014 Wisconsin

    I am kind of lazy, so I like to taste a beer, and give it a grade on my phone quick.
     
  11. utopiajane

    utopiajane Grand Pooh-Bah (3,982) Jun 11, 2013 New York
    Pooh-Bah

    The precise scoring had it's appeal but there's something to be said for how the rating comes out after you have observed each category. Without doing that the rating seems more arbitrary. I like the current system but I was happy as long as it was an option and I did use the precise feature more than once.
     
  12. johnInLA

    johnInLA Pooh-Bah (2,350) Jun 12, 2005 California
    Pooh-Bah Trader

    The main reason I don't write reviews is because when I sit down to an enjoy a beer, I want to do just that, enjoy it. I have tried writing reviews and found that it takes much of the enjoyment of out drinking a beer for me, turning it into a homework assignment.

    I also rarely read other people's reviews. So I don't feel like I'm taking without giving back.

    Rating on the other hand, is something I have been doing for years, on almost every beer I drink. I maintain my own spreadsheet with all my ratings going back many years. As a BA/Hobbyist, I enjoy crunching the numbers different ways to give me more insight on what I like and what I don't.

    I too enjoyed the single rating and the increased precision. I also think the new system will discourage some people from rating. But to some degree, I think that's the point. They are looking for ratings with a bit more thought to them. So people that don't want to provide the extra information will stop rating.

    Others, like myself, will adjust and keep rating.

    Since I maintain my own ratings, I don't rely on BA for my own data. Instead I rate here because I want to contribute to a site that I have enjoyed using for many years and has contributed so much to beer enthusiasts over the years.
     
    Canada_Dan and MLaVioletteJr like this.
  13. PapaGoose03

    PapaGoose03 Grand High Pooh-Bah (6,057) May 30, 2005 Michigan
    BA4LYFE Society Pooh-Bah

    It's not a 'new' rating system but the re-implementation of what was in place just two(?) years ago. This method forces you to think more about the beer rather than a seat-of-the-pants overall score (which I think if a study were done would show that most scores are overly generous), and then a precise score get's calculated from your input by the BA formula. (At least that's what I remember.)
     
  14. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    I can assure you that it won't. What's in place now has been our core system for nearly two decades.
     
  15. mikeburd1128

    mikeburd1128 Maven (1,409) Oct 28, 2011 New Jersey

    I'm not sure what you mean. I can remember plenty of changes to the system in the last few years. If I remember right, it was the addition of the quick rating, then the inclusion of quick ratings into the overall ratings of the beers. Then didn't we get rid of full reviews and attributes while adding precise ratings and then within a few weeks reinstate the full reviews? And now these most recent changes. I understand that maybe the overall way of calculating ratings hasn't changed, but there's been plenty of changes from a user's viewpoint.
     
  16. kevanb

    kevanb Pooh-Bah (2,705) Apr 4, 2011 Illinois
    Pooh-Bah Trader

    I'll address a few things here.

    First, each user weighs each attribute differently and that is encouraged, remember, one review won't drastically alter the overall scores of the beer, but it still equally as important as the others. Much like voting in a Presidential election.

    Secondly, yes, the BA algorithm is somewhat arbitrary, but does focus on the two most important attributes, taste and smell. If you find yourself tweaking to circumvent that, I don't see it as a huge issue and I do think that .25 intervals are better, and would help alleviate this problem.

    Thirdly, if you are at a beer tasting with 30+ beers, you should absolutely NOT being reviewing them. Palate fatigue is a big issue and unless you have multiple stomachs, I can't imagine you are getting anything more than 1-3oz of a single beer, and that is by no means enough to truly evaluate a beer properly. I'm sorry, it has to be said, it's even clearly laid out in the Beer 101 page for reviewing, that reviewing at festivals or from samplers is poor form. This is exactly what untappd is for.

    If the reviews actually mean something here, it's because people are taking the time to do this thoughtfully, not on a whim.
     
    Phocion and jmdrpi like this.
  17. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Obviously there's been tweaks and the introduction of other rating style layers over the years. And we learned a lot. But the core system has been in place for nearly two decades. That's simply a fact that I'm stating for clarity in the thread.

    Back to your initial comment that the rating system will most likely change again in 6 months. I can assure you that it won't.
     
  18. jmdrpi

    jmdrpi Grand High Pooh-Bah (8,989) Dec 11, 2008 Pennsylvania
    BA4LYFE Society Pooh-Bah

    Well said, I agree. Going back to the old system is a major positive. In my opinion, too many "5" scores seemed to be a problem with the quick ratings.
     
  19. jmdrpi

    jmdrpi Grand High Pooh-Bah (8,989) Dec 11, 2008 Pennsylvania
    BA4LYFE Society Pooh-Bah

    You shouldn't be rating beers on 2 oz pours.
     
    jasonmason and Phocion like this.
  20. sd123

    sd123 Pooh-Bah (2,073) Jan 15, 2012 Georgia
    Pooh-Bah

    If the problem was too many 5s with the quick rating system is forcing people to enter 4 numbers instead of 1 really going to change that? If a person wanted to give a beer a 5 its still easy to give it straight 5's across the board for an overall 5.

    The best thing about beeradvocate is that it has the most users of any beer site. More users means more reviews which means each individual rating has less impact. The cumulative averages on beeradvocate are significant because they come from the greatest number of users. I dont think discouraging some people from entering reviews because they "arent putting enough thought into them" is going to make the site better.
     
    iTunesUpdates likes this.
Thread Status:
Not open for further replies.