Weighted Beer Reviews

Discussion in 'BeerAdvocate Talk' started by defunksta, Sep 1, 2019.

  1. defunksta

    defunksta Zealot (589) Jan 18, 2019 Illinois
    Trader

    Curious if anyone else has considered weighting beer reviews? Some sort of system where the Top Reviewers have more weight than a regular review. In a world where a beer has thousands of reviews, maybe having some sort of weighted system would create more accurate rating scales. I also realize the downside to this would be an oligarchy over most beers, where the select few influence the beer's rating. Just a thought.
     
  2. bbtkd

    bbtkd Poo-Bah (2,745) Sep 20, 2015 South Dakota
    Society Trader

    Too many issues. For one, what criteria determines "Top Reviewers", since reviews are subjective and there is no right answer? And if top reviewers means number of reviews, then they probably DO weigh more! :wink:
     
  3. readyski

    readyski Aspirant (249) Jun 4, 2005 California
    Trader

    Hey some of us bystanders are no slouches either
     
  4. Scott17Taylor

    Scott17Taylor Meyvn (1,376) Oct 28, 2013 Iowa
    Trader

    Nah, over the course of many reviews everything averages out. I’m not too worried about the quality of the ratings here.
     
    Squire and dcotom like this.
  5. InVinoVeritas

    InVinoVeritas Devotee (430) Apr 16, 2012 Wisconsin

    What I'd most like to see with reviews is a separate rating by those that are BJCP. People that are either rating beers within stays they don't like, don't know how to score within style, or both, are giving good beers bad scores.
     
    SLeffler27 likes this.
  6. NickSMpls

    NickSMpls Meyvn (1,132) Nov 11, 2012 Washington
    Society Trader

    I'd stick with the principle that once a beer gets a significant number of reviews and ratings (say 100), it's highly likely for the beer's rating to stay pretty much where it is.

    Weighting reviews wouldn't change that unless the weighting was extremely heavy. And who wants to give those individuals that much influence ?

    Simple is better.
     
    Junior and PapaGoose03 like this.
  7. drtth

    drtth Poo-Bah (4,006) Nov 25, 2007 Pennsylvania

    Weighted beer reviews? Just restrict your view to reviews only and then sort by Top Reviewer. That gives you a list of reviews weighted by experience with reviewing.
     
  8. mudbug

    mudbug Defender (600) Mar 27, 2009 Oregon

    I am wholly in agreement. Just look at some lower rated beers like Hamms. If you averaged the written reviews it would be farther up the list of AALs. Rating only should be a fraction of reviews and Reviews with Karma should be multiplied.
     
  9. mudbug

    mudbug Defender (600) Mar 27, 2009 Oregon

    We already do this, Say you are looking for a lager because it's freeking hot, The Top list is useless but the AAL list or other lager lists are what you need to consider. You have weighted the rankings according to your situation. I do believe the ease of ranking makes it easier to downgrade without explanation.
     
  10. nc41

    nc41 Poo-Bah (1,944) Sep 25, 2008 North Carolina
    Trader

    Ratings are completely subjective, why weight it more to “top reviewers” , based on what? Volume? Could be it’s relevant, could be it’s not. See: the shitty reviews on AALs. How’s that rated to style? Are they all 1.8 level beers rated to their style? No they’re not. They’re being rated vs more flavorful beers like PAs and IPAs. Rated to style a really good AAL should dominate the ratings to style, over a mediocre IPA, but that’s certainly not close to being true.
     
  11. mudbug

    mudbug Defender (600) Mar 27, 2009 Oregon

    I've always said the truth lies in not the rating but the number of reviews, The truth is BAs in the vast majority only rate or review beers they like or are impressed by
    Top "reviewers" actually spend the effort to explain the rating raters do not.
     
    SLeffler27, superspak and stevepat like this.
  12. Premo88

    Premo88 Poo-Bah (1,831) Jun 6, 2010 Texas
    Society Trader

    What he said.
     
    Mindcrime1000 likes this.
  13. drtth

    drtth Poo-Bah (4,006) Nov 25, 2007 Pennsylvania

    Now suppose he's got some wrong assumptions about how the reviews/ratings numbers actually work...
     
    nc41 likes this.
  14. nc41

    nc41 Poo-Bah (1,944) Sep 25, 2008 North Carolina
    Trader

    So if every AAL is rated to style and they tend to scores a 2.5 or so, does this make the style amazingly consistent in it’s Mediocrity? There’s no doubt in my mind light lagers and Pils suffer greatly here looking at the ratings. Somewhere there should be an AAL that someone loves that's scores a 4.7 or so. Hell even I did it when I first started rating 10 years ago, it’s easy to lose perspective that it’s rated to style.
     
    drtth likes this.
  15. drtth

    drtth Poo-Bah (4,006) Nov 25, 2007 Pennsylvania

    Nobody asks us to “rate to style” on this site. We are not trained judges nor trained in the use of rating scales. You imposed that rating to style on yourself.

    You are assuming that a 3 always means the same thing even though the context changes. Not a safe assumption given the use of rating scales and the lack of statistical corrections for changes in context.
     
    ScaryEd likes this.
  16. nc41

    nc41 Poo-Bah (1,944) Sep 25, 2008 North Carolina
    Trader

    If you don’t rate to style then it renders any numerical number you attach to it a meaningless.
     
  17. drtth

    drtth Poo-Bah (4,006) Nov 25, 2007 Pennsylvania

    Meaningless? Not at all.

    It just doesn't mean the same thing it would mean if we were judging beer totally blind in a beer competition. In fact, for my purposes, it is more meaningful given what I'm looking for in a beer. This is, after all, a consumer-oriented site not a competition. So for my money, some weight should be given to the enjoyment factor while keeping style in mind.
     
    #17 drtth, Sep 1, 2019
    Last edited: Sep 1, 2019
    stevepat, ScaryEd and nc41 like this.
  18. ScaryEd

    ScaryEd Poo-Bah (1,944) Feb 19, 2012 New Hampshire

    I fully understand the importance of "rating to style". I get it. It's important for balanced ratings and it's how the pro judges do it.

    Here's the problem: I'm not an expert on beer. There are many styles I've had only a handful of beers from over the course of 7-8 years. How do I rate those to style? What about a style I've literally never had before?

    Also, if there's a style I don't like, say AAL's, am I really supposed to give the "least worst" version of that style a 4.75/5 even though it was an incredibly mediocre beer? "I don't really like this beer, but it's not a drain pour so I'm giving it 5/5" doesn't sound good to me.

    All that said, I'm not a beer expert. I just love beer and I love rating and reviewing them. I enjoy the vast majority of beers I try. I really just review because it's fun, and I mainly do it for myself. If my reviews are worthless because I don't truly rate to style, that's fine with me. I don't rate to get likes or notoriety. This is a fun site with a great system for beer lovers of any kind.

    IMO, people need to lighten up a bit. We all have preferences but we all love beer. Great time to be alive.
     
    beersampler6 and rudiecantfail like this.
  19. nc41

    nc41 Poo-Bah (1,944) Sep 25, 2008 North Carolina
    Trader

    In place of a numerical rating then I’d suggest a big green thumbs up or a big red thumbs down as to whether it’s a buy or not. No numbers, no worries about rating to style, you not comparing a lager to a big DIPA. Easy and simple, a buy or no buy, and other than that putting numerical rating to beers is a bit ridiculous if you don’t keep style in mind. Keeping in mind I don’t rate beers anymore for the most part, but when I look at the ratings I know how it works here, 100 ipa is a great beer, a 90 is mediocre, a 90 Pils is world class and an 80 Pils is still a good beer. AALs for the most part are all rated under 3, but we know what we like there. Its terribly slanted, we know that. So the rating thing is so over rated imo, a simple I like it, or I hate it is sufficient, or even a yellow middling grade would suffice.
     
    #19 nc41, Sep 2, 2019
    Last edited: Sep 2, 2019
  20. defunksta

    defunksta Zealot (589) Jan 18, 2019 Illinois
    Trader

    Maybe weighted reviews is not the right approach. Perhaps more filtering/sorting options would provide a similar result. If you could sort by experience, comments, length. I don't know, just more options, if you have any suggestions. I am all for review comments and likes (?which seems to be disabled right now).
    Sorting by top reviewers isn't a fix all because it gives an oligarchy approach where for most beers your a only reading a select few people's review from years and years ago. Many more recent reviews of younger reviews are more accurate.
     
  21. readyski

    readyski Aspirant (249) Jun 4, 2005 California
    Trader

    Or maybe that 3.75 pilsner is worth the extra mile but the 4.1 NEIPA is not
     
    nc41 likes this.
  22. nc41

    nc41 Poo-Bah (1,944) Sep 25, 2008 North Carolina
    Trader

    You’d have to be experienced on this site to know that curve exists. And it certainly does. Reviews here are a popularity contest, it’s hard /impossible for a simple lager even a great lager to even begin to reach average ipa status by the numbers. So what does that numerical rating mean then ?
     
  23. readyski

    readyski Aspirant (249) Jun 4, 2005 California
    Trader

    You just need your magical BA decoder ring. Certainly you have one right :wink:
     
  24. drtth

    drtth Poo-Bah (4,006) Nov 25, 2007 Pennsylvania

    It means that if you want to select or use the ratings to evaluate a Pils, look only within the Pils category and the ranking of those beers. If you want to select or use the ratings to evaluate a NEIPA, look only within the NEIPA category and the ranking of those beers. That effectively cancels out the popularity effect of one style over another.
     
    stevepat and PapaGoose03 like this.