Weighting of scores

A recent discussion of Pilsner Urquell triggered this thought. It is my understanding that currently all review scores count with the same weight towards the overall average for each beer, regardless of when those reviews were done. Using PU as the example, this means for PU that ratings from back in the day before PU switched to brown bottles and their cool shipment initiative began will have generally been on samples that were less than fresh and potentially not well protected from temperature and light before purchase. Many newer ratings will be on samples that were in closed packages and brown bottles that were shipped across the Atlantic in chilled condition.

This suggests the idea that it might be desirable to have a differential weighting over time on the contribution of ratings to the overall rating score. If a beer has declined or improved in quality there could then be an additional gradual change in the overall average over time that reflects things such as better or worse quality control, etc. As a different example, I know of at least one locally brewed Pale Ale that did a hop change in the their recipe and many folks think it is now a much better beer than reflected by its overall score.

This "moving average" would not require any new information beyond what already exists and would seem to provide a possible "smoothing" effect reflecting changes over time. It would also seem consistent with having the n small enough for new beers to be able to break into the Top 250 or into the year's best new beers list, etc.

 

Yes, good points both and with regard to the effect of old reviews, both better and worse would be equally affected so that the effects of something we might call the "Storm King envelope pushing" would be gradually reduced somewhat over time creating a more realistic view of its present status in the field. (Such dialog and exploration of the consequences of something like this weighting is part of the reason the OP was originally created as a comment rather than a feature request.)

 

True enough, and if the older a review gets the less weight it has, then the more the law of large numbers would push the average towards a more current representation of the its status relative to others in the style. Especially when it is being compared/contrasted with other, newer versions of the style that do not have an extended history.

 

It's an interesting assumption that the old guard, many of whom are no longer around, used to write better, more perceptive, more knowledgable reviews when they were new to the site. It's also a testable one which may be worth exploring. My personal experience tells me that as my experience with writing reviews has grown, so has my ability to do a sensory evaluation and communicate it.

It also tells me that newer site members have no monopoly on not knowing certain distinctions that are crucial to others. I've seen old reviews that obviously confused hop bitterness and skunking, and a fair number that equated hop bitterness with hop flavors when they are separate and separable things both on the palate and in the brewing process. I also recall being assured by old timers early on in my first year or so on the site that that there was a formula to tell us how bitter any beer would be. Indeed I was once venomously told how ignorant I was when trying to explain/understand how and why perceived bitterness seemed to be different from the number the formula gave.

So I just keep thinking, just as some older quite accurate scores would begin to gradually lose weight over time so to would some older scores that reflect misunderstanding and inaccuracy and since those numbers, in theory, mostly cancel each other out now there wouldn't seem to be much lost except an effect that clouds rhe current numbers with the weight of the way things used to be compared to the way they are now. The world of flavorful beer has changed even in a time frame as short as five years and we can argue we have more and better beers to chose from than we did as little as 10 years ago and that has a changing effect over time.

Two things a gradual decline with age in the weight of review scores in figuring the overall average would not do is to change the score we see when looking at the individual reviews or effect in any way the words we read and the value, learning and perceptiveness we find when reading those reviews.