Navigating heterogeneity in meta-analysis: methods for identification and management

Authors

  • Muhammad Ilmawan Medical Research Center, Deka Institute, Malang, Indonesia

DOI:

https://doi.org/10.69863/dim.2024.e269

Keywords:

Meta-analysis, heterogeneity, identification, management, statistical analysis

Abstract

Although meta-analysis is a powerful way to synthesize research findings from multiple studies, the problem of heterogeneity usually arises due to variation in study outcomes. Differences between studies regarding heterogeneity in results can arise from populations, interventions, outcome measures, and methodologies both within and between the studies. This article aims to provide an overview of the methods for identifying and dealing with heterogeneity in meta-analyses to ensure accurate and reliable conclusions. The article aims to describe the application of several statistical methods for detecting heterogeneity, namely the Q statistic and the I² statistic. The Q statistic is used to test whether observed variability in effect sizes exceeds chance expectations, while the I² statistic quantifies the proportion of variability due to heterogeneity. Other methods include the DerSimonian-Laird between-studies variance in random-effects models and the T and T² methods, which use both observed and expected information about effect size dispersion. Methods for dealing with heterogeneity are discussed, including choices between using fixed- versus random-effects models, and techniques for assessing and dealing with outliers using methods such as the Hedges technique. Additionally, the article explores methods to investigate sources of heterogeneity through subgroup analysis and meta-regression. Recognizing limitations such as residual heterogeneity, publication bias, and study quality is also important in making meta-analytical findings more robust. In conclusion, these methods enable researchers to more effectively address heterogeneity issues in meta-analyses, thereby providing more reliable and valid conclusions that contribute to evidence-based practice

Downloads

Download data is not yet available.

References

Wang XM, Zhang XR, Li ZH, et al. A brief introduction of meta-analyses in clinical practice and research. J Gene Med 2021;23(5):e3312.doi: 10.1002/jgm.3312. PMID: 33450104

Gurevitch J, Koricheva J, Nakagawa S, et al. Meta-analysis and the science of research synthesis. Nature 2018;555(7695):175-182.doi: 10.1038/nature25753. PMID: 29517004

Greco T, Zangrillo A, Biondi-Zoccai G, et al. Meta-analysis: pitfalls and hints. Heart Lung Vessel 2013;5(4):219-225.doi. PMID: 24364016

Yadav S. Challenges and concerns in the utilization of meta-analysis software: Navigating the landscape of scientific synthesis. Cureus 2024;16(1):e53322.doi: 10.7759/cureus.53322. PMID: 38435898

Ranganathan P, Aggarwal R. Study designs: Part 8 - Meta-analysis (I). Perspect Clin Res 2020;11(4):178-181.doi: 10.4103/picr.PICR_283_20. PMID: 33489837

Glass GV. Primary, secondary, and meta-analysis of research. Educational researcher 1976;5(10):3-8.doi: 10.3102/0013189X0050100. PMID:

Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21(11):1539-1558.doi: 10.1002/sim.1186. PMID: 12111919

Tong G, Guo G. Meta-analysis in sociological research: Power and heterogeneity. Sociol Methods Res 2022;51(2):566-604.doi: 10.1177/0049124119882479. PMID: 35754525

Linden AH, Honekopp J. Heterogeneity of research results: A new perspective from which to assess and promote progress in psychological science. Perspect Psychol Sci 2021;16(2):358-376.doi: 10.1177/1745691620964193. PMID: 33400613

Kulinskaya E, Hoaglin DC. On the Q statistic with constant weights in meta-analysis of binary outcomes. BMC Med Res Methodol 2023;23(1):146.doi: 10.1186/s12874-023-01939-z. PMID: 37344771

Cochran WG. The combination of estimates from different experiments. Biometrics 1954;10(1):101-129.doi: 10.2307/3001666. PMID:

van Aert RCM, van Assen M, Viechtbauer W. Statistical properties of methods based on the Q-statistic for constructing a confidence interval for the between-study variance in meta-analysis. Res Synth Methods 2019;10(2):225-239.doi: 10.1002/jrsm.1336. PMID: 30589219

Pereira TV, Patsopoulos NA, Salanti G, et al. Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity. Res Synth Methods 2010;1(2):149-161.doi: 10.1002/jrsm.13. PMID: 26061380

Huedo-Medina TB, Sanchez-Meca J, Marin-Martinez F, et al. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods 2006;11(2):193-206.doi: 10.1037/1082-989X.11.2.193. PMID: 16784338

von Hippel PT. The heterogeneity statistic I(2) can be biased in small meta-analyses. BMC Med Res Methodol 2015;15(1):35.doi: 10.1186/s12874-015-0024-z. PMID: 25880989

Thorlund K, Imberger G, Johnston BC, et al. Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses. PLoS One 2012;7(7):e39471.doi: 10.1371/journal.pone.0039471. PMID: 22848355

Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327(7414):557-560.doi: 10.1136/bmj.327.7414.557. PMID: 12958120

Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ 2007;335(7626):914-916.doi: 10.1136/bmj.39343.408449.80. PMID: 17974687

Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods 2016;7(1):55-79.doi: 10.1002/jrsm.1164. PMID: 26332144

DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7(3):177-188.doi: 10.1016/0197-2456(86)90046-2. PMID: 3802833

Zhai C, Guyatt G. Fixed-effect and random-effects models in meta-analysis. Chin Med J (Engl) 2024;137(1):1-4.doi: 10.1097/CM9.0000000000002814. PMID: 37612263

Jackson D, Law M, Barrett JK, et al. Extending DerSimonian and Laird's methodology to perform network meta-analyses with random inconsistency effects. Stat Med 2016;35(6):819-839.doi: 10.1002/sim.6752. PMID: 26423209

DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials 2015;45(Pt A):139-145.doi: 10.1016/j.cct.2015.09.002. PMID: 26343745

Bender R, Friede T, Koch A, et al. Methods for evidence synthesis in the case of very few studies. Res Synth Methods 2018;9(3):382-392.doi: 10.1002/jrsm.1297. PMID: 29504289

Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses. PLoS One 2013;8(7):e69930.doi: 10.1371/journal.pone.0069930. PMID: 23922860

Welton NJ, Soares MO, Palmer S, et al. Accounting for heterogeneity in relative treatment effects for use in cost-effectiveness models and value-of-information analyses. Med Decis Making 2015;35(5):608-621.doi: 10.1177/0272989X15570113. PMID: 25712447

Scammacca N, Roberts G, Stuebing KK. Meta-analysis with complex research designs: Dealing with dependence from multiple measures and multiple group comparisons. Rev Educ Res 2014;84(3):328-364.doi: 10.3102/0034654313500826. PMID: 25309002

Dettori JR, Norvell DC, Chapman JR. Fixed-Effect vs Random-Effects Models for Meta-Analysis: 3 Points to Consider. Global Spine J 2022;12(7):1624-1626.doi: 10.1177/21925682221110527. PMID: 35723546

Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc 2009;172(1):137-159.doi: 10.1111/j.1467-985X.2008.00552.x. PMID: 19381330

Partlett C, Riley RD. Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med 2017;36(2):301-317.doi: 10.1002/sim.7140. PMID: 27714841

Viechtbauer W, Cheung MW. Outlier and influence diagnostics for meta-analysis. Res Synth Methods 2010;1(2):112-125.doi: 10.1002/jrsm.11. PMID: 26061377

Kadlec D, Sainani KL, Nimphius S. With great power comes great responsibility: Common errors in meta-analyses and meta-regressions in strength & conditioning research. Sports Med 2023;53(2):313-325.doi: 10.1007/s40279-022-01766-0. PMID: 36208412

Lin L, Chu H, Hodges JS. Alternative measures of between-study heterogeneity in meta-analysis: Reducing the impact of outlying studies. Biometrics 2017;73(1):156-166.doi: 10.1111/biom.12543. PMID: 27167143

Hedges LV, Olkin I. Statistical methods for meta-analysis. Orlando, FL: Academic press; 1985.

Gress TW, Denvir J, Shapiro JI. Effect of removing outliers on statistical inference: implications to interpretation of experimental data in medical research. Marshall J Med 2018;4(2):9.doi: 10.18590/mjm.2018.vol4.iss2.9. PMID: 32923665

Prager EM, Chambers KE, Plotkin JL, et al. Improving transparency and scientific rigor in academic publishing. Brain Behav 2019;9(1):e01141.doi: 10.1002/brb3.1141. PMID: 30506879

Mikolajewicz N, Komarova SV. Meta-analytic methodology for basic research: A practical guide. Front Physiol 2019;10(1):203.doi: 10.3389/fphys.2019.00203. PMID: 30971933

Wang X, Piantadosi S, Le-Rademacher J, et al. Statistical considerations for subgroup analyses. J Thorac Oncol 2021;16(3):375-380.doi: 10.1016/j.jtho.2020.12.008. PMID: 33373692

Tipton E, Pustejovsky JE, Ahmadi H. A history of meta-regression: Technical, conceptual, and practical developments between 1974 and 2018. Res Synth Methods 2019;10(2):161-179.doi: 10.1002/jrsm.1338. PMID: 30589224

Glass GV. Integrating findings: the meta‐analysis of research. Rev Res Educ 1977;5(1):351‐379.doi: 10.3102/0091732x005001351. PMID:

Mathur MB, VanderWeele TJ. Meta-regression methods to characterize evidence strength using meaningful-effect percentages conditional on study characteristics. Res Synth Methods 2021;12(6):731-749.doi: 10.1002/jrsm.1504. PMID: 34196505

Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med 2002;21(11):1559-1573.doi: 10.1002/sim.1187. PMID: 12111920

Nair AS. Publication bias - Importance of studies with negative results! Indian J Anaesth 2019;63(6):505-507.doi: 10.4103/ija.IJA_142_19. PMID: 31263309

Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74(3):785-794.doi: 10.1111/biom.12817. PMID: 29141096

De Cassai A, Boscolo A, Zarantonello F, et al. Enhancing study quality assessment: an in-depth review of risk of bias tools for meta-analysis-a comprehensive guide for anesthesiologists. J Anesth Analg Crit Care 2023;3(1):44.doi: 10.1186/s44158-023-00129-z. PMID: 37932825

Russo MW. How to review a meta-analysis. Gastroenterol Hepatol (N Y) 2007;3(8):637-642.doi. PMID: 21960873

Downloads

Published

2024-08-13

How to Cite

Ilmawan, M. . (2024). Navigating heterogeneity in meta-analysis: methods for identification and management. Deka in Medicine, 1(2), e269. https://doi.org/10.69863/dim.2024.e269

Issue

Section

Editorial

Similar Articles

You may also start an advanced similarity search for this article.