Approaches for identifying and managing publication bias in meta-analysis

Authors

  • Jonny Karunia Fajar Medical Research Center, Deka Institute, Malang, Indonesia

DOI:

https://doi.org/10.69863/dim.v1i1.1

Keywords:

Publication bias, meta-analysis, systematic review, statistical techniques

Abstract

The consequences of publication bias in meta-analysis pose significant risks, potentially leading to erroneous conclusions within the meta-analytic framework. The objective of this article was to explore the methodologies for identifying publication bias and approaches for mitigating its effects. The techniques employed to detect publication bias can generally be distinguished into two major categories: graphical and statistical methodologies. Graphical approaches utilize techniques such as funnel plots and meta-plots, which visually depict the distribution of effect sizes and standard errors across studies. Statistical methods encompass various computations, including Fail-Safe N, rank correlation, Egger regression, tests for excess significance (TES), and selection models tailored for evaluating publication bias through quantitative analyses. The combination of these methods is recommended for a more comprehensive assessment, rather than relying on individual approaches. Methods for addressing publication bias include the trim and fill (T&F) method, Publication Error and True Effect Size Estimation (PET-PEESE) method, and the Weight-Function Model, each offering unique strategies for adjusting effect size estimates. The selection of these methods should consider the specific characteristics of the meta-analysis under consideration, ensuring the most appropriate approach is employed. Publication bias poses a significant risk in the field of meta-analysis, and selecting methods for its identification and mitigation requires comprehensive consideration

Downloads

Download data is not yet available.

References

Haidich AB. Meta-analysis in medical research. Hippokratia 2010;14(1):29-37.doi. PMID: 21487488

Sutton AJ, Higgins JP. Recent developments in meta-analysis. Stat Med 2008;27(5):625-650.doi: 10.1002/sim.2934. PMID: 17590884

Metelli S, Chaimani A. Challenges in meta-analyses with observational studies. Evid Based Ment Health 2020;23(2):83-87.doi: 10.1136/ebmental-2019-300129. PMID: 32139442

Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74(3):785-794.doi: 10.1111/biom.12817. PMID: 29141096

Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000;53(2):207-216.doi: 10.1016/s0895-4356(99)00161-4. PMID: 10729693

Mueller KF, Meerpohl JJ, Briel M, et al. Methods for detecting, quantifying, and adjusting for dissemination bias in meta-analysis are described. J Clin Epidemiol 2016;80(1):25-33.doi: 10.1016/j.jclinepi.2016.04.015. PMID: 27502970

Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001;54(10):1046-1055.doi: 10.1016/s0895-4356(01)00377-8. PMID: 11576817

Godavitarne C, Robertson A, Ricketts DM, et al. Understanding and interpreting funnel plots for the clinician. Br J Hosp Med (Lond) 2018;79(10):578-583.doi: 10.12968/hmed.2018.79.10.578. PMID: 30290744

Mayer EK, Bottle A, Rao C, et al. Funnel plots and their emerging application in surgery. Ann Surg 2009;249(3):376-383.doi: 10.1097/SLA.0b013e31819a47b1. PMID: 19247021

Simmonds M. Quantifying the risk of error when interpreting funnel plots. Syst Rev 2015;4(1):24.doi: 10.1186/s13643-015-0004-8. PMID: 25875027

Tang JL, Liu JL. Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol 2000;53(5):477-484.doi: 10.1016/s0895-4356(99)00204-8. PMID: 10812319

Terrin N, Schmid CH, Lau J, et al. Adjusting for publication bias in the presence of heterogeneity. Stat Med 2003;22(13):2113-2126.doi: 10.1002/sim.1461. PMID: 12820277

van Aert R, van Assen MALM, van den Akker OR, et al. The meta-plot: A graphical tool for interpreting the results of a meta-analysis. Zeitschrift für Psychologie 2023;23(1):65-78.doi: 10.1027/2151-2604/a000513. PMID:

Lin L. Graphical augmentations to sample-size-based funnel plot in meta-analysis. Res Synth Methods 2019;10(3):376-388.doi: 10.1002/jrsm.1340. PMID: 30664834

Poorolajal J, Mahmoodi M, Majdzadeh R, et al. Metaplot: a novel stata graph for assessing heterogeneity at a glance. Iran J Public Health 2010;39(2):102-104.doi. PMID: 23113013

Poorolajal J, Noornejad S. Metaplot: A new Stata module for assessing heterogeneity in a meta-analysis. PLoS One 2021;16(6):e0253341.doi: 10.1371/journal.pone.0253341. PMID: 34181682

Ferguson CJ, Brannick MT. Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychol Methods 2012;17(1):120-128.doi: 10.1037/a0024445. PMID: 21787082

Rosenthal R. The file drawer problem and tolerance for null results. Psychological bulletin 1979;86(3):638.doi. PMID:

Rosenberg MS. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 2005;59(2):464-468.doi. PMID: 15807430

Fragkos KC, Tsagris M, Frangos CC. Publication bias in meta-analysis: Confidence intervals for rosenthal's fail-safe number. Int Sch Res Notices 2014;2014(1):825383.doi: 10.1155/2014/825383. PMID: 27437470

Dowdy A, Hantula DA, Travers JC, et al. Meta-analytic methods to detect publication bias in behavior science research. Perspect Behav Sci 2022;45(1):37-52.doi: 10.1007/s40614-021-00303-0. PMID: 35342865

Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Online Library: Wiley; 2008.

Persaud R. Misleading meta-analysis. "Fail safe N" is a useful mathematical measure of the stability of results. BMJ 1996;312(7023):125.doi: 10.1136/bmj.312.7023.125. PMID: 8555918

Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109):629-634.doi: 10.1136/bmj.315.7109.629. PMID: 9310563

Peters JL, Sutton AJ, Jones DR, et al. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006;295(6):676-680.doi: 10.1001/jama.295.6.676. PMID: 16467236

Schwab S, Kreiliger G, Held L. Assessing treatment effects and publication bias across different specialties in medicine: a meta-epidemiological study. BMJ Open 2021;11(9):e045942.doi: 10.1136/bmjopen-2020-045942. PMID: 34521659

Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994;50(4):1088-1101.doi. PMID: 7786990

Gjerdevik M, Heuch I. Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis. BMC Med Res Methodol 2014;14(1):109.doi: 10.1186/1471-2288-14-109. PMID: 25245217

Lin L, Chu H, Murad MH, et al. Empirical comparison of publication bias tests in meta-analysis. J Gen Intern Med 2018;33(8):1260-1267.doi: 10.1007/s11606-018-4425-7. PMID: 29663281

Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings. Clin Trials 2007;4(3):245-253.doi: 10.1177/1740774507079441. PMID: 17715249

Tsilidis KK, Panagiotou OA, Sena ES, et al. Evaluation of excess significance bias in animal studies of neurological diseases. PLoS Biol 2013;11(7):e1001609.doi: 10.1371/journal.pbio.1001609. PMID: 23874156

Ioannidis JP. Excess significance bias in the literature on brain volume abnormalities. Arch Gen Psychiatry 2011;68(8):773-780.doi: 10.1001/archgenpsychiatry.2011.28. PMID: 21464342

van Assen MA, van Aert RC, Wicherts JM. Meta-analysis using effect size distributions of only statistically significant studies. Psychol Methods 2015;20(3):293-309.doi: 10.1037/met0000025. PMID: 25401773

Simonsohn U, Nelson LD, Simmons JP. p-curve and effect size: Correcting for publication bias using only significant results. Perspect Psychol Sci 2014;9(6):666-681.doi: 10.1177/1745691614553988. PMID: 26186117

van Aert RC, Wicherts JM, van Assen MA. Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve. Perspect Psychol Sci 2016;11(5):713-729.doi: 10.1177/1745691616650874. PMID: 27694466

van Aert RCM, Wicherts JM, van Assen M. Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS One 2019;14(4):e0215052.doi: 10.1371/journal.pone.0215052. PMID: 30978228

Scofield JE, Buchanan EM, Kostic B. A meta-analysis of the survival-processing advantage in memory. Psychon Bull Rev 2018;25(3):997-1012.doi: 10.3758/s13423-017-1346-0. PMID: 28744766

Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000;56(2):455-463.doi: 10.1111/j.0006-341x.2000.00455.x. PMID: 10877304

Weinhandl ED, Duval S. Generalization of trim and fill for application in meta-regression. Res Synth Methods 2012;3(1):51-67.doi: 10.1002/jrsm.1042. PMID: 26061999

Shi L, Lin L. The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses. Medicine (Baltimore) 2019;98(23):e15987.doi: 10.1097/MD.0000000000015987. PMID: 31169736

Cano G, Suarez-Roca H, Bonilla E. Alterations of excitatory amino acid receptors in the brain of manganese-treated mice. Mol Chem Neuropathol 1997;30(1):41-52.doi: 10.1007/BF02815149. PMID: 9138428

Stanley TD. Limitations of PET-PEESE and Other Meta-Analysis Methods. Social Psychological and Personality Science 2017;8(5):581-591.doi: 10.1177/1948550617693062. PMID:

Iyengar S, Greenhouse JB. Selection models and the file drawer problem. Statist Sci 1988;3(1):109-117.doi: 10.1214/ss/1177013012. PMID:

Silliman NP. Nonparametric classes of weight functions to model publicationbias. Biometrika 1997;84(4):909-918.doi. PMID:

McShane BB, Bockenholt U, Hansen KT. Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspect Psychol Sci 2016;11(5):730-749.doi: 10.1177/1745691616662243. PMID: 27694467

Rufibach K. Selection models with monotone weight functions in meta analysis. Biom J 2011;53(4):689-704.doi: 10.1002/bimj.201000240. PMID: 21567443

Citkowicz M, Vevea JL. A parsimonious weight function for modeling publication bias. Psychol Methods 2017;22(1):28-41.doi: 10.1037/met0000119. PMID: 28252998

Downloads

Published

2024-03-25

How to Cite

Fajar, J. (2024). Approaches for identifying and managing publication bias in meta-analysis. Deka in Medicine, 1(1), e865. https://doi.org/10.69863/dim.v1i1.1

Issue

Section

Editorial

Similar Articles

You may also start an advanced similarity search for this article.