A megastudy or mega-study is a research study in which a large number of different treatments or interventions are tested at the same time, on the same sample or similar samples, using a common outcome measure, and using the same experimental protocol.[1][2][3]
This research design ensures the outcomes across interventions are comparable. Additionally, due to generous inclusion of various interventions in the study, megastudies may be less prone to publication bias, where the interventions expected to be effective are more likely to be studied and the interventions found to be ineffective are underreported due to the file-drawer problem.[4]
Megastudy examples
editMany-lab studies
editThe megastudy technique can be combined with the many-labs approach, so that teams of researchers from across the planet conduct the same experiment locally.[3][7]
Megastudy criticisms
edit- Statistical power: While the overall megastudy sample size may be large, the sample size per intervention may be relatively small, leading to underpowered designs with wide confidence intervals. As a result, while interventions may be comparable, their relative ranking by outcome measure may be noisy. Increased sample size can help address this issue.[8]
- Lack of theory: On the one hand, the megastudy technique may be considered a form of "fishing expedition" for what interventions have strongest effect on the outcome measure, without much theory building.[8] On the other hand, such theory-free approach can help uncover unexpected findings, mitigate publication bias, and form new theories.[4]
References
edit- ^ a b Milkman, K.L., Gromet, D., Ho, H. et al. Megastudies improve the impact of applied behavioural science. Nature 600, 478–483 (2021). https://doi.org/10.1038/s41586-021-04128-4
- ^ Angela L Duckworth, Katherine L Milkman, A guide to megastudies, PNAS Nexus, Volume 1, Issue 5, November 2022, https://doi.org/10.1093/pnasnexus/pgac214
- ^ a b c Doell, K.C. Megastudies to test the efficacy of behavioural interventions. Nat Rev Psychol 2, 263 (2023). https://doi.org/10.1038/s44159-023-00174-z
- ^ a b c Tkachenko, Y., Jedidi, K. A megastudy on the predictability of personal information from facial images: Disentangling demographic and non-demographic signals. Sci Rep 13, 21073 (2023). https://doi.org/10.1038/s41598-023-42054-9
- ^ Milkman, Katherine L., et al. "A 680,000-person megastudy of nudges to encourage vaccination in pharmacies." Proceedings of the National Academy of Sciences 119.6 (2022): e2115126119. https://doi.org/10.1073/pnas.2115126119
- ^ Voelkel, Jan G., et al. "Megastudy identifying successful interventions to strengthen Americans’ democratic attitudes." Northwestern University: Evanston, IL, USA (2022). https://www.ipr.northwestern.edu/documents/working-papers/2022/wp-22-38.pdf
- ^ Pavlov, Yuri G.; Adamian, Nika; Appelhoff, Stefan; Arvaneh, Mahnaz; Benwell, Christopher S. Y.; Beste, Christian; Bland, Amy R.; Bradford, Daniel E.; Bublatzky, Florian; Busch, Niko A.; Clayson, Peter E.; Cruse, Damian; Czeszumski, Artur; Dreber, Anna; Dumas, Guillaume (2021-11-01). "#EEGManyLabs: Investigating the replicability of influential EEG experiments". Cortex. 144: 213–229. doi:10.1016/j.cortex.2021.03.013. hdl:1885/295623. ISSN 0010-9452.
- ^ a b Collins J (2022-05-27). "Megastudy scepticism". jasoncollins.blog. Retrieved 2023-12-13.