Averagely Perfect ESB - Poll #33 - General Water Profile

Perfect timing!

http://brulosophy.com/2017/05/01/wa...d-when-ratio-is-the-same-exbeeriment-results/

(TLDR: Could not achieve statistical significance for people trying to determine a difference between two beers with the same Cl / SO4 ratio, but varying amounts)

 

Specifically, in one beer the salt additions were calculated to result in 200 ppm sulfate and 100 ppm chloride, while in the other they were calculated to result in 51 ppm sulfate and 27 ppm chloride. Therefore, one of the beers was "medium/high" by our classification, and the other was "low/low."

I would have designed the experiment a bit differently, but the results are certainly interesting. I really, really wish the Brulosophy crew would run replication experiments, though. Absent that, you have to take everything with a huge grain of salt.

 

If it were just one huge grain of salt, it would surely mess up the ratio. Just saying.

 

agreed

 

Don't forget, Brulosophy xbmts are single-tailed with the single goal of confirming a correlation, such that when they do find significance, they are 95% confident in the result. However the opposite is NOT true, i.e., when they do NOT find significance, there is NOT 95% confidence that there is NOT a correlationship. This point might make sense when you think hard about it.

That, and they're only one data point. A group of non-tasters not tasting something. Go figure.

 

next poll
https://www.beeradvocate.com/commun...centrations-for-chloride-and-sulfates.508377/

 

Agreed as well, hence the specific wording I used in my post!

They do make statements to the same effect, but not nearly as explicit or as prominent.

 

Not enough replication of experiments and slavish devotion to p-values is a problem in a number of research fields. Still, I appreciate their work

 

I don't care about the lack of replications, but the whole thing seems mostly pointless given the number of tasters they are able to recruit. Almost all the effects they are trying to detect are small which means they need a large sample to be able to say something from a statistical point of view. I'm not sure I have ever seen one of their "experiments" find a statistically significant difference. Then combine that with the "non-tasters" that @dmtaylor mentions and I can't find too much use in the qualitative responses.

That was a lot to say that I like the idea, but the implementation leaves a lot to be desired

 

Replications help mitigate concerns about anomalies due to inadequate controls either during brewing or evaluating. If you ever listen to Experimental Brewing podcasts, the IGORs help serve this purpose.

As for number of evaluations, you do what you can. What I would like to see is some extrapolation similar to a power test, e.g. "22% of the evaluators detected a difference. We would need X # of participants for this to be significant."

 

I respect the Brulosophy gang and what they are trying to accomplish. They mean well, and they are visible advocates for science within a hobby that previously was so chock full of old wives tales and perpetuated Papazianisms that it wasn't even funny anymore.

My primary criticisms are that with such tiny sample sizes of only 15 or 20 tasters, they should never be shooting for 95% confidence. A more reasonable 80% confidence (p=0.2) would be most appropriate methinks. If at some later time they can reliably grab 50 or 60 tasters for every xbmt, then this could be raised up reasonably to 85 or 90% confidence. But 95% is far too lofty a goal for right now.

Denny & Drew's Experimental Brewing IGOR experiments are similarly run but maybe not quite as popular yet, but they run the distinct advantage over Brulosophy in that the same experiments are run independently by at least 6 or 8 different brewers across the world. Inevitably, they always find 1 or 2 sets of data that are total outliers with the rest, and appropriately, this data is largely set aside and disregarded, while the rest is lumped up, and a more realistic and repeatable result can be reported. Unfortunately, these experiments again will often result in more questions rather than answers.

Set p to a more reasonable 0.2, and then things look a lot more meaningful overall IMO.

Cheers all.