Navigating heterogeneity in meta-analysis: methods for identification and management
DOI:
https://doi.org/10.69863/dim.2024.e269Keywords:
Meta-analysis, heterogeneity, identification, management, statistical analysisAbstract
Although meta-analysis is a powerful way to synthesize research findings from multiple studies, the problem of heterogeneity usually arises due to variation in study outcomes. Differences between studies regarding heterogeneity in results can arise from populations, interventions, outcome measures, and methodologies both within and between the studies. This article aims to provide an overview of the methods for identifying and dealing with heterogeneity in meta-analyses to ensure accurate and reliable conclusions. The article aims to describe the application of several statistical methods for detecting heterogeneity, namely the Q statistic and the I² statistic. The Q statistic is used to test whether observed variability in effect sizes exceeds chance expectations, while the I² statistic quantifies the proportion of variability due to heterogeneity. Other methods include the DerSimonian-Laird between-studies variance in random-effects models and the T and T² methods, which use both observed and expected information about effect size dispersion. Methods for dealing with heterogeneity are discussed, including choices between using fixed- versus random-effects models, and techniques for assessing and dealing with outliers using methods such as the Hedges technique. Additionally, the article explores methods to investigate sources of heterogeneity through subgroup analysis and meta-regression. Recognizing limitations such as residual heterogeneity, publication bias, and study quality is also important in making meta-analytical findings more robust. In conclusion, these methods enable researchers to more effectively address heterogeneity issues in meta-analyses, thereby providing more reliable and valid conclusions that contribute to evidence-based practice
Downloads
References
Wang XM, Zhang XR, Li ZH, et al. A brief introduction of meta-analyses in clinical practice and research. J Gene Med 2021;23(5):e3312.doi: 10.1002/jgm.3312. PMID: 33450104
Gurevitch J, Koricheva J, Nakagawa S, et al. Meta-analysis and the science of research synthesis. Nature 2018;555(7695):175-182.doi: 10.1038/nature25753. PMID: 29517004
Greco T, Zangrillo A, Biondi-Zoccai G, et al. Meta-analysis: pitfalls and hints. Heart Lung Vessel 2013;5(4):219-225.doi. PMID: 24364016
Yadav S. Challenges and concerns in the utilization of meta-analysis software: Navigating the landscape of scientific synthesis. Cureus 2024;16(1):e53322.doi: 10.7759/cureus.53322. PMID: 38435898
Ranganathan P, Aggarwal R. Study designs: Part 8 - Meta-analysis (I). Perspect Clin Res 2020;11(4):178-181.doi: 10.4103/picr.PICR_283_20. PMID: 33489837
Glass GV. Primary, secondary, and meta-analysis of research. Educational researcher 1976;5(10):3-8.doi: 10.3102/0013189X0050100. PMID:
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21(11):1539-1558.doi: 10.1002/sim.1186. PMID: 12111919
Tong G, Guo G. Meta-analysis in sociological research: Power and heterogeneity. Sociol Methods Res 2022;51(2):566-604.doi: 10.1177/0049124119882479. PMID: 35754525
Linden AH, Honekopp J. Heterogeneity of research results: A new perspective from which to assess and promote progress in psychological science. Perspect Psychol Sci 2021;16(2):358-376.doi: 10.1177/1745691620964193. PMID: 33400613
Kulinskaya E, Hoaglin DC. On the Q statistic with constant weights in meta-analysis of binary outcomes. BMC Med Res Methodol 2023;23(1):146.doi: 10.1186/s12874-023-01939-z. PMID: 37344771
Cochran WG. The combination of estimates from different experiments. Biometrics 1954;10(1):101-129.doi: 10.2307/3001666. PMID:
van Aert RCM, van Assen M, Viechtbauer W. Statistical properties of methods based on the Q-statistic for constructing a confidence interval for the between-study variance in meta-analysis. Res Synth Methods 2019;10(2):225-239.doi: 10.1002/jrsm.1336. PMID: 30589219
Pereira TV, Patsopoulos NA, Salanti G, et al. Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity. Res Synth Methods 2010;1(2):149-161.doi: 10.1002/jrsm.13. PMID: 26061380
Huedo-Medina TB, Sanchez-Meca J, Marin-Martinez F, et al. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods 2006;11(2):193-206.doi: 10.1037/1082-989X.11.2.193. PMID: 16784338
von Hippel PT. The heterogeneity statistic I(2) can be biased in small meta-analyses. BMC Med Res Methodol 2015;15(1):35.doi: 10.1186/s12874-015-0024-z. PMID: 25880989
Thorlund K, Imberger G, Johnston BC, et al. Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses. PLoS One 2012;7(7):e39471.doi: 10.1371/journal.pone.0039471. PMID: 22848355
Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327(7414):557-560.doi: 10.1136/bmj.327.7414.557. PMID: 12958120
Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ 2007;335(7626):914-916.doi: 10.1136/bmj.39343.408449.80. PMID: 17974687
Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods 2016;7(1):55-79.doi: 10.1002/jrsm.1164. PMID: 26332144
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7(3):177-188.doi: 10.1016/0197-2456(86)90046-2. PMID: 3802833
Zhai C, Guyatt G. Fixed-effect and random-effects models in meta-analysis. Chin Med J (Engl) 2024;137(1):1-4.doi: 10.1097/CM9.0000000000002814. PMID: 37612263
Jackson D, Law M, Barrett JK, et al. Extending DerSimonian and Laird's methodology to perform network meta-analyses with random inconsistency effects. Stat Med 2016;35(6):819-839.doi: 10.1002/sim.6752. PMID: 26423209
DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials 2015;45(Pt A):139-145.doi: 10.1016/j.cct.2015.09.002. PMID: 26343745
Bender R, Friede T, Koch A, et al. Methods for evidence synthesis in the case of very few studies. Res Synth Methods 2018;9(3):382-392.doi: 10.1002/jrsm.1297. PMID: 29504289
Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses. PLoS One 2013;8(7):e69930.doi: 10.1371/journal.pone.0069930. PMID: 23922860
Welton NJ, Soares MO, Palmer S, et al. Accounting for heterogeneity in relative treatment effects for use in cost-effectiveness models and value-of-information analyses. Med Decis Making 2015;35(5):608-621.doi: 10.1177/0272989X15570113. PMID: 25712447
Scammacca N, Roberts G, Stuebing KK. Meta-analysis with complex research designs: Dealing with dependence from multiple measures and multiple group comparisons. Rev Educ Res 2014;84(3):328-364.doi: 10.3102/0034654313500826. PMID: 25309002
Dettori JR, Norvell DC, Chapman JR. Fixed-Effect vs Random-Effects Models for Meta-Analysis: 3 Points to Consider. Global Spine J 2022;12(7):1624-1626.doi: 10.1177/21925682221110527. PMID: 35723546
Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc 2009;172(1):137-159.doi: 10.1111/j.1467-985X.2008.00552.x. PMID: 19381330
Partlett C, Riley RD. Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med 2017;36(2):301-317.doi: 10.1002/sim.7140. PMID: 27714841
Viechtbauer W, Cheung MW. Outlier and influence diagnostics for meta-analysis. Res Synth Methods 2010;1(2):112-125.doi: 10.1002/jrsm.11. PMID: 26061377
Kadlec D, Sainani KL, Nimphius S. With great power comes great responsibility: Common errors in meta-analyses and meta-regressions in strength & conditioning research. Sports Med 2023;53(2):313-325.doi: 10.1007/s40279-022-01766-0. PMID: 36208412
Lin L, Chu H, Hodges JS. Alternative measures of between-study heterogeneity in meta-analysis: Reducing the impact of outlying studies. Biometrics 2017;73(1):156-166.doi: 10.1111/biom.12543. PMID: 27167143
Hedges LV, Olkin I. Statistical methods for meta-analysis. Orlando, FL: Academic press; 1985.
Gress TW, Denvir J, Shapiro JI. Effect of removing outliers on statistical inference: implications to interpretation of experimental data in medical research. Marshall J Med 2018;4(2):9.doi: 10.18590/mjm.2018.vol4.iss2.9. PMID: 32923665
Prager EM, Chambers KE, Plotkin JL, et al. Improving transparency and scientific rigor in academic publishing. Brain Behav 2019;9(1):e01141.doi: 10.1002/brb3.1141. PMID: 30506879
Mikolajewicz N, Komarova SV. Meta-analytic methodology for basic research: A practical guide. Front Physiol 2019;10(1):203.doi: 10.3389/fphys.2019.00203. PMID: 30971933
Wang X, Piantadosi S, Le-Rademacher J, et al. Statistical considerations for subgroup analyses. J Thorac Oncol 2021;16(3):375-380.doi: 10.1016/j.jtho.2020.12.008. PMID: 33373692
Tipton E, Pustejovsky JE, Ahmadi H. A history of meta-regression: Technical, conceptual, and practical developments between 1974 and 2018. Res Synth Methods 2019;10(2):161-179.doi: 10.1002/jrsm.1338. PMID: 30589224
Glass GV. Integrating findings: the meta‐analysis of research. Rev Res Educ 1977;5(1):351‐379.doi: 10.3102/0091732x005001351. PMID:
Mathur MB, VanderWeele TJ. Meta-regression methods to characterize evidence strength using meaningful-effect percentages conditional on study characteristics. Res Synth Methods 2021;12(6):731-749.doi: 10.1002/jrsm.1504. PMID: 34196505
Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med 2002;21(11):1559-1573.doi: 10.1002/sim.1187. PMID: 12111920
Nair AS. Publication bias - Importance of studies with negative results! Indian J Anaesth 2019;63(6):505-507.doi: 10.4103/ija.IJA_142_19. PMID: 31263309
Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74(3):785-794.doi: 10.1111/biom.12817. PMID: 29141096
De Cassai A, Boscolo A, Zarantonello F, et al. Enhancing study quality assessment: an in-depth review of risk of bias tools for meta-analysis-a comprehensive guide for anesthesiologists. J Anesth Analg Crit Care 2023;3(1):44.doi: 10.1186/s44158-023-00129-z. PMID: 37932825
Russo MW. How to review a meta-analysis. Gastroenterol Hepatol (N Y) 2007;3(8):637-642.doi. PMID: 21960873
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Muhammad Ilmawan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.