Thoughts on the latest updates? Top 250, etc.

I think you are likely bang on. Ratings without reviews are inherently less considered on average, and have massive skew now I am seeing the statistical impact. That is the number 1 thing I love about the new system - it has excluded all the noise from the data

 

@Todd Here is my biggest problem with the update. You lost a TON of data when creating this list now. For years you were not requiring reviews and most people didn't really want to write them. No one is going to go back through hundreds of "ticks" and add reviews. They would be wrong even if they did unless they took really detailed notes. If you want to make people write reviews because you feel it is the true BA way that is fine. Require it going forward and you will get your wish. Don't ignore all of the previous data that you have gathered because you are not going to get it back.

Also, I don't really think this changes anything. People who just want to tick are going to write stupid meaningless reviews. Some people on here are awesome and write long meaningful reviews. You can't force people to do that. I hope I am wrong but I think you are about to see a huge drop in quality on the reviews. I think you just made a lot more work for the mods who already have a lot of BS to deal with that they shouldn't have to.

What you have now is a list of a bunch of mediocre, dated beer. Looking at the beer that dropped off and what was added it is just wrong. Beer has changed a lot even in the year since I joined this site. The new breweries are not properly represented in this list anymore and it is a shame. I was trying to drink all of the beers on the top 250 list because it was a good way to try all of the best beer that was out there and keep current without researching every brewery and what they are doing. I was at 235/250. Now I am at 185/250 and I have probably 10-15 that I drank and didn't rate. Those beers are nowhere close to the ones that were lost. The list is just wrong now and I am definitely not going to try and drink all of the top 250 beers anymore. Looking at the list I have very little interest in the beers that I haven't tried already.

 

So what, all this means is that people who might be a little bit lazy or might be too busy to write a full review might rate a beer 0.13 higher than those who write a "full" review. Do you really think this is statistically significant? Also, don't forget if this 0.13 elevation is allowed across the board, then there is no significant advantage to any individual beer! Again, I will state, simply allowing more statistical data can only improve BA averages and scores, not damage them. If it affects your favorite beer, it will most likely affect all beers. The only statistically significant effect is the changing rules over time and how this will effect old beers versus new beers.

 

Show us the data that supports your claims.

 

Well according to you more ratings are better and so with enough ratings that 0.13 becomes statistically significant.

 

I always thought BA ratings where better because it was more real review oriented, not just ticking a had like Untappd (most the people on that site don't know the difference between a Pils or a Saison to begin with), or writing one nonsensical sentence on RB. It was not always the shear numbers that validated a score for me, but the number of well thought out reviews written. BA was always behind in shear numbers, and I don't think that is a bad thing. I have been to many bottle shares, and there are probably many thousands of beers that I have not review on BA because of that. I don't think people will ever be able to review everything they have ever drank. I do think it is worth "hads" being a small percentage of the overall rating, if more people are inclined to do that. I just don't think you always have to rate everything you have ever tasted...getting long winded, and on some tangents so I am stopping now.

Also, on a general side note, Westy 12 is awesome, no reason why it should not be in the top whatever, for any reason

 

BTW the 0.13 would not be across the board given that there are clear stylistic biases in ratings, with strongly flavored beer styles having higher ratings that more subtly favored beers.

 

This WONT only happen going back and reviewing previous beers. I can see reviews in the future that read something like - this is a beer. I really like this beer. I think you'll like this beer too. Let us know what you think. 5/5.

 

Naw, nothing wrong with that. It infects some of us periodically despite the best efforts of friends and family. I often catch flack for how often I'm found saying "show me the data." :-)

 

I almost tagged you to see what your thoughts were. Interesting read.

 

It makes more sense to throw out all reviews and ratings and have everyone start over.

@Todd I know I already messaged you, but why not give the user the option to view the lists baised on ratings+ reviews or reviews alone. This new system is actually agrivating to me and useless to most of us.

 

Actually it is epically statistically significant
the probability that the difference in mean scores is an error is 3.14E-51....

 

I see that you have already made assumptions about me personally and probably assume that I am a novice simply because I choose not to write 150+ characters when I "rate" a beer. If you know anything about statistics then you should know that requiring 150+ characters does not make your data statistically more significant.

If you want to compare pedigrees, I have a PhD in experimental particle physics and have spent years studying math and physics, and currently work at a DoE National lab on LI, NY. If you don't believe me ask my friends here on BA. Although I did not like statistical mechanics and thermodynamics that much, I do understand the basics statistical principals well enough.

Asking for my quick review to count does not help me personally in any way, since my rating will be meaningless when measured amongst the hundreds or thousands of other ratings and reviews on BA. But it does help me when I'm looking for beers that I have never had before, where dozens of other BA users have had them, but only a tiny minority have reviewed them. In this circumstance, which is very common for new or local or not popular beers, I much prefer higher quantity of statistical data than lower "quality" reviews.

 

There is a report button to deal with that sort of shenanigans... Just sayn

 

The same bias will be applied to both quick ratings and full reviews. More statistics means a better statistical result. Do you have proof to the contrary?

 

I personally love the update, reminds me of the BA I missed.

 

While you make some good points, if someone does write a low quality review (of which I am certainly guilty of), you are free to determine if that review is well written or of no quality. For me it is hard to make that decision base one a score with no context at all.

 

Cool - then we can have an actual conversation about the stats if you like. This is not a personal attack, but an observation based on the data set at hand (the old vs new ratings).

Please don't take this the wrong way - I am not saying your ratings are crap.

I am sure your ratings are considered and thoughtful, but on the whole when all the ratings with and without the review filter are compared there is a right shift in the distribution, and a very significant skew.

None of this proves the phenomenon will persist - I have serious doubts as I suspect people will start putting in crappy reviews of 150 words that are dull and stock, but as the data stands now, people who rate vs actually review do skew high.

 

And the mods have all day to determine who is actually writing a review, perhaps a poorly done albeit honest one, and a lengthy fraudulent one? It'll be a pure witch hunt that will take far longer for the mods to sort out.

 

Beyond a certain level increasing the n does not give a meaningfully better statistical result. Increasing the n for reviews is a perfectly reasonable strategy and given the loss of information provided when one allows ratings only it becomes less easy to interpret the result.

Edit: BTW, its not incumbent on me to develop a proof. You're the one whose claiming degradation of results. As I said earlier, "Show me the data."