I'd just add that I don't think the creatine study linked is likely to be trustworthy. One standard deviation on cognitive tests is an effect ten times larger than regularly feeding school children in developing countries (source: https://academic.oup.com/wbro/article/39/2/159/7613189). It is also has several times the effect on memory as not having had enough sleep (https://www.sciencedirect.com/science/article/pii/S0149763424003981). Almost everyone would feel very bad within a few weeks after going vegetarian if it were true. As the p-value is so low in the study that it cannot be noise (it's not the standard underpowered study leading to massively exaggerated effects), there was clearly either some serious error in the study design/implementation, an error in data analysis, or fraud.
Thanks a lot for noting this. I first heard about this study forever ago from Rob Wiblin and Paul Christiano on an 80,000 hours podcast. It just sort of stuck around my brain and solidified into being true without me ever looking too deeply into it. I don't know enough about statistics to adjudicate between what you say and what Christiano say about the study. Here is what he had to say about it in a facebook post (https://www.facebook.com/alexei.andreev.3/posts/1573007142808719)
* The effect is very large (~1 standard deviation amongst college students). This is not a statistical fluke or garden of forking paths, p <0.0001 and they report the most straightforward measure possible on the most straightforward design.
* There are no other good studies in vegetarians (Benton and Donohoe find no effect... from 4 days of a maintenance dose). The effect isn't that sexy to replicate and it takes 12 weeks, so I wouldn't be too surprised if there has been no high-quality replication attempt. (See Gwern's meta-analysis: https://www.gwern.net/Creatine).
* The biological mechanism is plausible. Creatine supplementation is known to have (very) large effects on muscle tissue, creatine has a plausible role in the brain, and vegetarians consume way less creatine than omnivores. Fixing nutrient deficiencies generally seems like the most plausible way to boost cognitive performance."
I agree with you, it seems very unlikely to be doing as much as the study claims. I am still open to the idea that it has some slight cognitive benefits.
I found the transcript of the podcast and Paul says he gives creatine a 5-10% chance of having a big effect on the cognition of vegetarians.
I agree with Christiano's take that the study design looks good on paper, it's not forking paths/random noise, and it's biologically plausible for an effect to exist.
My reasoning comes from my experience as labour and education economist. Vanishingly few things have effects bigger than 0.1 to 0.2 standard deviations on cognitive outcomes in well designed studies. Health interventions with effects far larger than that are not things which go under the radar clinically. It would be something on the scale of a serious B12 deficiency, if that. That means reasonable Bayesian priors should be very sceptical of the study.
I am sympathetic to the idea that some unnoticed nutritional deficiencies may have far smaller effects than 1SD that are worth an individual addressing. There's some decent quality evidence that iodine deficiency has that effect (although I remember one well designed RCT, I think by Esther DuFlo or Abhijit Banerjee, couldn't find it). With creatine, it reminds me of an AstralCodexTen post on drugs/supplements where the benefits seem plausible in a cost-benefit analysis, but so do 50 other things with imperfect evidence that also probably don't work. It's a tough one to think about.
Be careful, raising my daughter non-vegan is partially responsible for my leaving veganism!
It has been a life saver from a piece of mind and convenience perspective especially when travelling. Also one thing to consider is giving non-vegan foods to prevent allergies e.g. eggs, shrimp. Just don't buy too ethical or you might start eating them yourself!
Ha! As someone with a massive appetite and who hates food waste, I imagine I will be eating some of my kid's non-vegan leftovers. My goal is not to win some ultimate vegan purity test, but to live my values in a pragmatic way.
And that's a good point about the allergies, I haven't thought about that piece
I'd just add that I don't think the creatine study linked is likely to be trustworthy. One standard deviation on cognitive tests is an effect ten times larger than regularly feeding school children in developing countries (source: https://academic.oup.com/wbro/article/39/2/159/7613189). It is also has several times the effect on memory as not having had enough sleep (https://www.sciencedirect.com/science/article/pii/S0149763424003981). Almost everyone would feel very bad within a few weeks after going vegetarian if it were true. As the p-value is so low in the study that it cannot be noise (it's not the standard underpowered study leading to massively exaggerated effects), there was clearly either some serious error in the study design/implementation, an error in data analysis, or fraud.
Thanks a lot for noting this. I first heard about this study forever ago from Rob Wiblin and Paul Christiano on an 80,000 hours podcast. It just sort of stuck around my brain and solidified into being true without me ever looking too deeply into it. I don't know enough about statistics to adjudicate between what you say and what Christiano say about the study. Here is what he had to say about it in a facebook post (https://www.facebook.com/alexei.andreev.3/posts/1573007142808719)
"Some thoughts on why I find Rae 2003 (https://www.ncbi.nlm.nih.gov/.../PMC1691485/pdf/14561278.pdf) unusually credible:
* The effect is very large (~1 standard deviation amongst college students). This is not a statistical fluke or garden of forking paths, p <0.0001 and they report the most straightforward measure possible on the most straightforward design.
* There are no other good studies in vegetarians (Benton and Donohoe find no effect... from 4 days of a maintenance dose). The effect isn't that sexy to replicate and it takes 12 weeks, so I wouldn't be too surprised if there has been no high-quality replication attempt. (See Gwern's meta-analysis: https://www.gwern.net/Creatine).
* The biological mechanism is plausible. Creatine supplementation is known to have (very) large effects on muscle tissue, creatine has a plausible role in the brain, and vegetarians consume way less creatine than omnivores. Fixing nutrient deficiencies generally seems like the most plausible way to boost cognitive performance."
And here is the podcast link if anyone is curious to check it out. You can search the transcript for creatine to get right to that section: https://80000hours.org/podcast/episodes/paul-christiano-a-message-for-the-future/#transcript
I agree with you, it seems very unlikely to be doing as much as the study claims. I am still open to the idea that it has some slight cognitive benefits.
I found the transcript of the podcast and Paul says he gives creatine a 5-10% chance of having a big effect on the cognition of vegetarians.
https://80000hours.org/podcast/episodes/paul-christiano-a-message-for-the-future/#transcript
I agree with Christiano's take that the study design looks good on paper, it's not forking paths/random noise, and it's biologically plausible for an effect to exist.
My reasoning comes from my experience as labour and education economist. Vanishingly few things have effects bigger than 0.1 to 0.2 standard deviations on cognitive outcomes in well designed studies. Health interventions with effects far larger than that are not things which go under the radar clinically. It would be something on the scale of a serious B12 deficiency, if that. That means reasonable Bayesian priors should be very sceptical of the study.
I am sympathetic to the idea that some unnoticed nutritional deficiencies may have far smaller effects than 1SD that are worth an individual addressing. There's some decent quality evidence that iodine deficiency has that effect (although I remember one well designed RCT, I think by Esther DuFlo or Abhijit Banerjee, couldn't find it). With creatine, it reminds me of an AstralCodexTen post on drugs/supplements where the benefits seem plausible in a cost-benefit analysis, but so do 50 other things with imperfect evidence that also probably don't work. It's a tough one to think about.
Fascinating! Your take makes even more sense to me now. Thanks for those additional details.
Be careful, raising my daughter non-vegan is partially responsible for my leaving veganism!
It has been a life saver from a piece of mind and convenience perspective especially when travelling. Also one thing to consider is giving non-vegan foods to prevent allergies e.g. eggs, shrimp. Just don't buy too ethical or you might start eating them yourself!
Ha! As someone with a massive appetite and who hates food waste, I imagine I will be eating some of my kid's non-vegan leftovers. My goal is not to win some ultimate vegan purity test, but to live my values in a pragmatic way.
And that's a good point about the allergies, I haven't thought about that piece
And congratulations btw
Thank you!