Update: Top Rated Beer Lists (m; minimum ratings required to be considered)

Blog Discussion in 'BeerAdvocate Talk' started by Todd, Oct 28, 2014.

Thread Status:
Not open for further replies.
  1. Todd

    Todd Founder (13,518) Aug 23, 1996 Finland
    STAFF Mod Team Society Pooh-Bah

    Following feedback from the Top Rated Beers: US State update, this morning I applied a significant update to our algorithm that impacts how they'll work going forward.

    Lists are based on Weighted Rank (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C

    where:

    R = average rating for the beer
    v = number of votes (ratings) for the beer
    m = minimum ratings required to be considered
    C = the mean across the current list​

    In the past, m was an arbitrary number. Like 100 for the Top 250, 1000 for Beers of Fame, and 10 for US regions, countries, and beer styles. This may have worked when we had less ratings and the datasets were smaller, but as our data grows these arbitrary numbers show their ... well, arbitrariness.

    So what's m now? It's still the "minimum ratings required to be considered," however, it's represented as the average number of ratings within the dataset; instead of a number pulled from our asses. Meaning that m will now change based on the dataset being examined. This essentially helps to level the playing field, generates a much more accurate and dynamic (and arguably useful) listing, and provides consistency across the lists.

    Examples (m):
    Note: Some listings (actually very few) may be shorter than others or appear to be dominated by a brewery. This might seem unfair, but the dataset needs to be considered.

    Domination Example:
    Dogfish Head currently dominates to the Top Rated Beers: Delaware listing, but looking a the dataset shows only 12 breweries in the state with many having a low beer average and/or number of ratings. In this case, it is what it is, however, there's nothing stopping another brewery making the list if they get more ratings.

    Short List Example:
    Top Rated Beers: North Dakota currently only has 7 beers listed, however, the dataset is only examining 9 breweries; 5 of which have no ratings. The list will only grow as more people rate beers from the state.​

    Anyway. @Jason and I just reviewed every list. We're extremely happy with the results.

    And thanks to @drtth and @treznor for suggesting the change.

    Cheers!
     
    barflybastard, Strix, DIM and 18 others like this.
Thread Status:
Not open for further replies.