It's Time to Embrace Ratings

Blog Discussion in 'BeerAdvocate Talk' started by Todd, Jan 10, 2022.

Thread Status:
Not open for further replies.
  1. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    There's been lot of talk over the years about beer ratings (1-5 and less than 150 characters) without reviews (150 characters or more). Some have suggested that those who just rate don't give it much thought (they're just ticking), that ratings are probably significantly higher versus those attached to reviews as a result, that we should give reviews more weight, or not count ratings at all.

    Let's take a look by pulling some numbers...

    Historical
    These numbers reflect when we had an app, which brought in a ton of ratings and reviews. What's a ton? From May 2014 to July 2016 (app lifespan), the app and site pulled in over 3,200,000 ratings and reviews with 86% being ratings.

    (Yeah, I know. We need an app. Along with being able to rate a beer multiple times, attaching a photo to your rating with ease, adding serving type back, and much more.)

    Anyway, when we look at everything, there's a 0.07 differences between the average rating w/o reviews and the average rating w/ reviews, and 55.7% more ratings over reviews.

    Total Ratings: 10,221,853
    Total Average: 3.9
    Average Rating w/o Reviews: 3.92
    Number of Ratings w/o Reviews: 7,080,810
    Average Rating w/ Reviews: 3.85
    Number of Reviews: 3,141,043

    2019 (The Before Times)
    Back when people were going out all the time, there was a 0.05 difference between averages with 58.8% more ratings over reviews.

    Total Ratings: 423,544
    Total Average: 3.98
    Average Rating w/o Reviews: 4.0
    Number of Ratings w/o Reviews: 299,932
    Average Rating w/ Reviews: 3.95
    Number of Reviews: 123,612

    2020 (Pandemic)
    We saw a significant drop in overall site activity due to the pandemic, the gap between averages closed a bit with a 0.03 difference, and users posted 44.6% more ratings over reviews.

    Total Ratings: 297,068 (down 29.9%)
    Total Average: 3.99
    Average Rating w/o Reviews: 4.0
    Number of Ratings w/o Reviews: 191,123 (down 36.3%)
    Average Rating w/ Reviews: 3.97
    Number of Reviews: 105,945 (down 14.43%)

    2021 (Pandemic Limbo)
    The number of ratings and reviews were both down with 40.1% more ratings over reviews, averages went down for both, and the difference between averages widened back to 0.05.

    Total Ratings: 270,169 (down 9%)
    Total Average: 3.98
    Average Rating w/o Reviews: 4.0
    Number of Ratings w/o Reviews: 169,003 (down 11.6%)
    Average Rating w/ Reviews: 3.95
    Number of Reviews: 101,166 (down 4.5%)

    Ratings Need a Hug
    The ability to rate a beer without reviewing has been around for over a decade, it's clearly more popular amongst users, aggregated ratings are very useful, and ratings act as a gateway to reviewing and contributing elsewhere on the site. Without ratings, we'd have a lot less user activity, less beer data, and many beers wouldn't get noticed on the site.

    Like posting in the forums, posting a beer review requires the user to feel comfortable articulating their thoughts. We can't force people to review. We've tried. And we've failed. It needs to happen organically. Same with rating, but to a much lesser extent. So, I plan on making some adjustments to encourage more users to log in and rate beers, while continuing to explore ways that we can guide users toward reviewing overtime without it feeling intimidating.

    I'm already projecting over 300,000 total ratings in 2022 without changing a thing. That said, I'd like to aim for 500,000. Open to any ideas that you might have, but confident that we can collectively make it happen by embracing ratings instead of hating on them.

    Thanks for reading.
     
  2. Rug

    Rug Grand Pooh-Bah (3,454) Aug 20, 2018 Massachusetts
    Pooh-Bah Trader

    I don't often post in NBW and WBAYDN threads, but maybe we could encourage posters in there to at the very least give the beer they post a quick rating
     
  3. DIM

    DIM Grand Pooh-Bah (4,788) Sep 28, 2006 Pennsylvania
    BA4LYFE Society Pooh-Bah Trader

    I think the possibility of getting some kind of feedback would help generate reviews. Just a simple "like" would be sufficient.

    Tagging other users in rates or reviews?
     
  4. SFACRKnight

    SFACRKnight Grand Pooh-Bah (3,348) Jan 20, 2012 Colorado
    Pooh-Bah Trader

    I use this site to talk about beer. It is my Facebook. Or meta. Or wtf ever. Here are some reasons why I have little interest in rating Or reviewing a beer...

    I don't feel like over thinking. I want to drink the beer and not focus on the why's of the experience. Why do I think this beer is better than Or worse than another beer within the same style.

    Ease of access, @Todd you are right in that an app has and would continue to facilitate this part of BA. I am lazy. I don't even try to post pictures any more due to laziness. If rating and reviewing were easier to access I would do it more often. Same with pictures. An app could facilitate both issues.

    Sometimes I have issues articulating what it is that I am tasting. A subforum of how to taste, issues we run into with descriptors etc could be helpful. I promise not to snark it up. If I felt more confident with my descriptions I would likely start up again.

    Just talking out loud as to the reasons I don't do it as often as I should. I really don't care for personal data points and was using them as a tool to help hone my homebrewing. Can't fix something you don't perceive right? Now that my kids are of the age that I need to spend that time with them I don't brew much any more. I may step back into it soon, but for now it is what it is I suppose.
     
  5. BigIronH

    BigIronH Grand Pooh-Bah (3,762) Oct 31, 2019 Michigan
    Pooh-Bah Trader

    We’ve got your word not to “snark it up”, now we just need the word of the dozen other users who are bound to do so…:wink:

    Cheers.
     
  6. Shanex

    Shanex Grand Pooh-Bah (4,960) Dec 10, 2015 France
    Mod Team Society Pooh-Bah Trader

    This about sums it up for me.

    I mostly post on the forums, often in WBAYDN. Sometimes in NBW. Have rated 457 beers and reviewed about half of them.

    And I do it less and less.

    I’m mostly here to enjoy the site, the discussions, the beers and the camaraderie.

    To not overthink it is a key point, BA is designed to make us learn about beers, breweries and also make fellow new beer geeks friends (I have met three venerable advocates and traded beers with each of them).

    I just enjoy the site for what it is. I can’t think of a better forum to discuss anything beer related and lord knows it’s a wide range of topics.

    I’m happy here. If I run out of new beers to rate or review, I’ll just buy some old reliable, go post it in the daily thread and that’s good enough for me.
     
  7. zid

    zid Grand Pooh-Bah (3,132) Feb 15, 2010 New York
    BA4LYFE Society Pooh-Bah Trader

    @Todd , Who would be your primary targeted group for any initiatives to increase ratings?
    - existing users who already rate
    - existing users who do not rate
    - people w/o an account currently

    One would think incentives can drive results. "What's in it for me?" needs a good answer... whatever that may be. I am ignorant about how badges on Untappd work, but I am assuming that they set more obtainable goals far more clearly and concretely (and on-going) for the user than the karma system here. "Tracking your beer" might not be incentive enough with Untappd in the space. Any reward also has to work well with whatever burden is being placed on the user. If the burden is high, the incentive/reward needs to match it... and rewards like badges/karma are immaterial. A low burden can drive more interaction. If someone rated a beer on BeerAdvocate, would they currently have to enter 5 different numbers (look, smell, etc.) or can they just enter 1 number?
     
  8. GratefulBeerGuy

    GratefulBeerGuy Pooh-Bah (2,918) May 20, 2006 New Hampshire
    Pooh-Bah Trader

    I'm in it for reviewing. BA for me, since 5/20/06 has been the place I rate and share my thoughts on the beers I have. It's more like my own catalog of beer, with exactly the aspects I see worthy of putting into words. I love this site for that, more than anything else. I think rating without a review really doesn't hold much weight. That's just my feeling. A rating without at least, a few words of what's good, what's bad doesn't mean anything to me. Thank you for what you do here at BeerAdvocate.
     
  9. bret27

    bret27 Grand Pooh-Bah (3,064) Mar 10, 2009 California
    Pooh-Bah Trader

    [​IMG]

    I used to review most beers on here. Then I rated everything to essentially log and keep track. I then switched over to untappd. I double-logged everything for a while but that got tedious
     
    MTKATL, Ranbot, eppCOS and 8 others like this.
  10. donspublic

    donspublic Grand Pooh-Bah (3,552) Aug 4, 2014 Texas
    BA4LYFE Society Pooh-Bah Trader

    Honestly, most people are not going to be comfortable spending the time to rate a beer. When I first started, I lingered a while, then started giving numbers, then said fuck it, no one knows me, I don't really give a shit what they think and I started writing reviews. The main reason for that was I wanted to go back and see what I liked about the beer or didn't like. I think my % of reviews to ratings has increased now that I drink more from home, but I am not drinking as many different beers as I did when I first got into this. One thing that might get people to give you more of a review would be the usage of tags. For example for look, there would be a list of tags they could select. For smell, taste another set of tags. So basically a review would look like below:
    Look: #clear #2fingerhead #goodretention #excellentlacing #golden
    Smell: #coconut #ripepineapple
    Taste: #pineapple #citrus #coconut
    Mouthfeel: #lowbitterness #dry
    Review: like this beer because it was easy to drink and was tropical in nature, and I am a sucker for anything with sabro

    I know this isn't something that most would like, but it gives you a feel for what the beer is like with little effort on the reviewers part. Honestly I am a shit reviewer, I review for myself and if you looked at my reviews, they could be decompiled into tags like above. One advantage of this, with the usage of tags, you could then offer a search that could go against these tags. For example if I know I like the pineapple/citrus/coconut thing I can search out for it.
     
  11. bubseymour

    bubseymour Grand Pooh-Bah (4,800) Oct 30, 2010 Maryland
    Pooh-Bah Trader

    @Todd agree with others, I think if you marketed/sold it to new comers and existing members alike with “here on Beer Advocate we provide you the tools to record tasting ratings and notes (then explain all of the various bullet points as to how useful the data stored will be to the individual). I think you know most of the pros from the forum comments over the years but we can always elaborate again and share enhancement ideas.
     
    Whyteboar, FBarber, Rug and 2 others like this.
  12. chipawayboy

    chipawayboy Pooh-Bah (2,181) Oct 26, 2007 Massachusetts
    Pooh-Bah Trader

    Reviewing a beer on BA is not intuitive - in order to entice others it would certainly help to have a search/rate interface right on the homepage. Perhaps this is too radical and expensive to code -- but it's the most obvious solution. Personally - I'm in the untappd camp since 2019 - given the ease and ability to post photos quickly but I have often crossed ref to BA if I'm on the hunt for a beer -- doing it less and less these days. BTW - thx for sharing the data- particularly surprised at the minimal avg. score delta between ratings for those w/ and w/out a review.
     
  13. elNopalero

    elNopalero Grand Pooh-Bah (5,822) Oct 14, 2009 Michigan
    Pooh-Bah Trader

    I think we all need a hug!

    One of my BA goals for this year is to contribute by reviewing more--and posting my reviews in a timely manner. My "system" is not at all efficient. I take/save notes to write up/post down the road, ideally. In practice, it leaves me with a backlog of things to post and edit. I could streamline this by composing and posting in BA, true, but I am not always online or in a situation where that makes sense.

    While these reviews are a contribution to the site, they are also for me to keep track of what I've had, which in turn helps me track what works and doesn't work for me.

    So, thinking about how I approach reviewing, how to make the site user-friendly/efficient, and how to set BA apart from Untappd, along with what @donspublic wrote, what about a menu with a limited set of options? "Like" or "dislike" plus an overall score might be the bare minimum, along with some overly general and deliberately vague descriptors for flavor/aroma/appearance: "hoppy," "sweet," "roasty," and whatnot.
     
  14. MNAle

    MNAle Initiate (0) Sep 6, 2011 Minnesota

    If you really want to embrace rating, then you need to make it a rating, not a half-review.

    Look -Smell-Taste-Feel-Overall without any words of explanation is no better (or at best, very little better) than a simple 1-5 rating.

    IOW, add a quick 1-5 rating without L-S-T-F-O. Probably leave a place for a short comment or two.

    Leave the current system for those who want to at least say something more, full fledged review or not.

    BTW, I ignore ratings. And it has nothing to do with any of the aspects you gathered stats for, @Todd.

    It is because I am not interested in merely someone's view on whether or not THEY liked the beer. I want to read the words for some explanation of WHY they liked (or didn't like) the beer. For example, if someone doesn't like an English-style barley wine because it is too malty, well, I know what to do with their rating.

    If I am looking up a beer on BA, I first hide ratings and then sort by "top reviewers." Then, I scan for high and low numbers and read.

    BTW, I consider the ratings on UT to be less than useless. Most don't add any comment at all, and it is too tedious to search for those than do.
     
    #14 MNAle, Jan 10, 2022
    Last edited: Jan 10, 2022
  15. mudbug

    mudbug Pooh-Bah (1,762) Mar 27, 2009 Oregon
    Pooh-Bah

    I really don't care as I ignore ratings, if it helps the site great.
     
    2beerdogs, mactrail, beergoot and 3 others like this.
  16. MNAle

    MNAle Initiate (0) Sep 6, 2011 Minnesota

    Once I got over the mental hurdle of not having the finely-tuned olfaction and palate some reviewers have, and just started describing what I smelled and tasted within my simple smell and taste vocabulary, it wasn't that hard.

    Just describe what YOU smell and taste. Don't worry if you can't pick out the specific hop being used or yeast strain. If you've never tasted a dragon fruit, who cares? Maybe it just tastes like a pear to you.
     
  17. maximum12

    maximum12 Grand Pooh-Bah (4,686) Jan 21, 2008 Minnesota
    Pooh-Bah Trader

    I've enjoyed writing reviews for well over a decade now. They are incredibly useful to me as a record of what I've had & how I've felt about it in the moment (& sometimes what kind of socks I was wearing). I also often dial up other user's opinions at times before I buy a beer; reviews have saved me from buying infected and/or trash beer, which I appreciate.

    That said, there are a lot of barriers to significantly increasing reviews. People have already mentioned some. One big one is venue. I very rarely review beers on draft as I'd rather enjoy the atmosphere & people I'm with than tap away on my phone like the geek I am. I know a lot of people feel the same. And almost doubling the ratings year-over-year seems unrealistic on the face of it.

    As for increasing reviews...I don't see it happening in a major way. By highlighting the benefits one could encourage a small percentage to jump on the bandwagon & I like the 'rewards' angle noted above & the ability to react to reviews or tag other users. I miss the 'serving type' because that was also part of my research in looking into new beers.

    To summarize my unhelpful post: I believe BA would need to create major incentives or make reviews hugely more useful to the browsing public to make any noticeable increase in reviews. A free box of beer for rater of the month? Or for the most likes on a review (though that could be manipulated)? Increased 'status' for those who rate X beers a month?
     
  18. jonphisher

    jonphisher Grand Pooh-Bah (3,850) Aug 9, 2015 New Jersey
    Society Pooh-Bah Trader

    Most of us, if not all of us who participate in NBW do this. The times I post in WBAYDN I try to as well.
     
  19. BillAfromSoCal

    BillAfromSoCal Pooh-Bah (2,415) Aug 24, 2020 California
    Society Pooh-Bah

    Ditto...A mere numeric rating means little to me. I use the review to discern something about how thoughtful and qualified the rater is in addition to see if their written description is a match for what I am looking for in a beer. However,I am actually surprised at how closely the average review score is to a rating score. It may suggest to me that I need to rethink how often I minimize the value of ratings while focusing on reviews.

    I would value some way of commenting on reviews, either with a thumbs up or better yet, a written comment either supporting or not supporting the reviewer's observations.
     
  20. BigIronH

    BigIronH Grand Pooh-Bah (3,762) Oct 31, 2019 Michigan
    Pooh-Bah Trader

    For me personally, a rating is a review without words. I score the beer just as I would in a review, and many others do as well which is made obvious by the numbers.
     
    RMW66, LXIXME, Mista_Carta and 9 others like this.
Thread Status:
Not open for further replies.