Approaches for identifying and managing publication bias in meta-analysis
DOI:
https://doi.org/10.69863/dim.v1i1.1Keywords:
Publication bias, meta-analysis, systematic review, statistical techniquesAbstract
The consequences of publication bias in meta-analysis pose significant risks, potentially leading to erroneous conclusions within the meta-analytic framework. The objective of this article was to explore the methodologies for identifying publication bias and approaches for mitigating its effects. The techniques employed to detect publication bias can generally be distinguished into two major categories: graphical and statistical methodologies. Graphical approaches utilize techniques such as funnel plots and meta-plots, which visually depict the distribution of effect sizes and standard errors across studies. Statistical methods encompass various computations, including Fail-Safe N, rank correlation, Egger regression, tests for excess significance (TES), and selection models tailored for evaluating publication bias through quantitative analyses. The combination of these methods is recommended for a more comprehensive assessment, rather than relying on individual approaches. Methods for addressing publication bias include the trim and fill (T&F) method, Publication Error and True Effect Size Estimation (PET-PEESE) method, and the Weight-Function Model, each offering unique strategies for adjusting effect size estimates. The selection of these methods should consider the specific characteristics of the meta-analysis under consideration, ensuring the most appropriate approach is employed. Publication bias poses a significant risk in the field of meta-analysis, and selecting methods for its identification and mitigation requires comprehensive consideration
References
Haidich AB. Meta-analysis in medical research. Hippokratia 2010;14(1):29-37.doi. PMID: 21487488
Sutton AJ, Higgins JP. Recent developments in meta-analysis. Stat Med 2008;27(5):625-650.doi: 10.1002/sim.2934. PMID: 17590884
Metelli S, Chaimani A. Challenges in meta-analyses with observational studies. Evid Based Ment Health 2020;23(2):83-87.doi: 10.1136/ebmental-2019-300129. PMID: 32139442
Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74(3):785-794.doi: 10.1111/biom.12817. PMID: 29141096
Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000;53(2):207-216.doi: 10.1016/s0895-4356(99)00161-4. PMID: 10729693
Mueller KF, Meerpohl JJ, Briel M, et al. Methods for detecting, quantifying, and adjusting for dissemination bias in meta-analysis are described. J Clin Epidemiol 2016;80(1):25-33.doi: 10.1016/j.jclinepi.2016.04.015. PMID: 27502970
Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001;54(10):1046-1055.doi: 10.1016/s0895-4356(01)00377-8. PMID: 11576817
Godavitarne C, Robertson A, Ricketts DM, et al. Understanding and interpreting funnel plots for the clinician. Br J Hosp Med (Lond) 2018;79(10):578-583.doi: 10.12968/hmed.2018.79.10.578. PMID: 30290744
Mayer EK, Bottle A, Rao C, et al. Funnel plots and their emerging application in surgery. Ann Surg 2009;249(3):376-383.doi: 10.1097/SLA.0b013e31819a47b1. PMID: 19247021
Simmonds M. Quantifying the risk of error when interpreting funnel plots. Syst Rev 2015;4(1):24.doi: 10.1186/s13643-015-0004-8. PMID: 25875027
Tang JL, Liu JL. Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol 2000;53(5):477-484.doi: 10.1016/s0895-4356(99)00204-8. PMID: 10812319
Terrin N, Schmid CH, Lau J, et al. Adjusting for publication bias in the presence of heterogeneity. Stat Med 2003;22(13):2113-2126.doi: 10.1002/sim.1461. PMID: 12820277
van Aert R, van Assen MALM, van den Akker OR, et al. The meta-plot: A graphical tool for interpreting the results of a meta-analysis. Zeitschrift für Psychologie 2023;23(1):65-78.doi: 10.1027/2151-2604/a000513. PMID:
Lin L. Graphical augmentations to sample-size-based funnel plot in meta-analysis. Res Synth Methods 2019;10(3):376-388.doi: 10.1002/jrsm.1340. PMID: 30664834
Poorolajal J, Mahmoodi M, Majdzadeh R, et al. Metaplot: a novel stata graph for assessing heterogeneity at a glance. Iran J Public Health 2010;39(2):102-104.doi. PMID: 23113013
Poorolajal J, Noornejad S. Metaplot: A new Stata module for assessing heterogeneity in a meta-analysis. PLoS One 2021;16(6):e0253341.doi: 10.1371/journal.pone.0253341. PMID: 34181682
Ferguson CJ, Brannick MT. Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychol Methods 2012;17(1):120-128.doi: 10.1037/a0024445. PMID: 21787082
Rosenthal R. The file drawer problem and tolerance for null results. Psychological bulletin 1979;86(3):638.doi. PMID:
Rosenberg MS. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 2005;59(2):464-468.doi. PMID: 15807430
Fragkos KC, Tsagris M, Frangos CC. Publication bias in meta-analysis: Confidence intervals for rosenthal's fail-safe number. Int Sch Res Notices 2014;2014(1):825383.doi: 10.1155/2014/825383. PMID: 27437470
Dowdy A, Hantula DA, Travers JC, et al. Meta-analytic methods to detect publication bias in behavior science research. Perspect Behav Sci 2022;45(1):37-52.doi: 10.1007/s40614-021-00303-0. PMID: 35342865
Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Online Library: Wiley; 2008.
Persaud R. Misleading meta-analysis. "Fail safe N" is a useful mathematical measure of the stability of results. BMJ 1996;312(7023):125.doi: 10.1136/bmj.312.7023.125. PMID: 8555918
Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109):629-634.doi: 10.1136/bmj.315.7109.629. PMID: 9310563
Peters JL, Sutton AJ, Jones DR, et al. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006;295(6):676-680.doi: 10.1001/jama.295.6.676. PMID: 16467236
Schwab S, Kreiliger G, Held L. Assessing treatment effects and publication bias across different specialties in medicine: a meta-epidemiological study. BMJ Open 2021;11(9):e045942.doi: 10.1136/bmjopen-2020-045942. PMID: 34521659
Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994;50(4):1088-1101.doi. PMID: 7786990
Gjerdevik M, Heuch I. Improving the error rates of the Begg and Mazumdar test for publication bias in fixed effects meta-analysis. BMC Med Res Methodol 2014;14(1):109.doi: 10.1186/1471-2288-14-109. PMID: 25245217
Lin L, Chu H, Murad MH, et al. Empirical comparison of publication bias tests in meta-analysis. J Gen Intern Med 2018;33(8):1260-1267.doi: 10.1007/s11606-018-4425-7. PMID: 29663281
Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings. Clin Trials 2007;4(3):245-253.doi: 10.1177/1740774507079441. PMID: 17715249
Tsilidis KK, Panagiotou OA, Sena ES, et al. Evaluation of excess significance bias in animal studies of neurological diseases. PLoS Biol 2013;11(7):e1001609.doi: 10.1371/journal.pbio.1001609. PMID: 23874156
Ioannidis JP. Excess significance bias in the literature on brain volume abnormalities. Arch Gen Psychiatry 2011;68(8):773-780.doi: 10.1001/archgenpsychiatry.2011.28. PMID: 21464342
van Assen MA, van Aert RC, Wicherts JM. Meta-analysis using effect size distributions of only statistically significant studies. Psychol Methods 2015;20(3):293-309.doi: 10.1037/met0000025. PMID: 25401773
Simonsohn U, Nelson LD, Simmons JP. p-curve and effect size: Correcting for publication bias using only significant results. Perspect Psychol Sci 2014;9(6):666-681.doi: 10.1177/1745691614553988. PMID: 26186117
van Aert RC, Wicherts JM, van Assen MA. Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve. Perspect Psychol Sci 2016;11(5):713-729.doi: 10.1177/1745691616650874. PMID: 27694466
van Aert RCM, Wicherts JM, van Assen M. Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS One 2019;14(4):e0215052.doi: 10.1371/journal.pone.0215052. PMID: 30978228
Scofield JE, Buchanan EM, Kostic B. A meta-analysis of the survival-processing advantage in memory. Psychon Bull Rev 2018;25(3):997-1012.doi: 10.3758/s13423-017-1346-0. PMID: 28744766
Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000;56(2):455-463.doi: 10.1111/j.0006-341x.2000.00455.x. PMID: 10877304
Weinhandl ED, Duval S. Generalization of trim and fill for application in meta-regression. Res Synth Methods 2012;3(1):51-67.doi: 10.1002/jrsm.1042. PMID: 26061999
Shi L, Lin L. The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses. Medicine (Baltimore) 2019;98(23):e15987.doi: 10.1097/MD.0000000000015987. PMID: 31169736
Cano G, Suarez-Roca H, Bonilla E. Alterations of excitatory amino acid receptors in the brain of manganese-treated mice. Mol Chem Neuropathol 1997;30(1):41-52.doi: 10.1007/BF02815149. PMID: 9138428
Stanley TD. Limitations of PET-PEESE and Other Meta-Analysis Methods. Social Psychological and Personality Science 2017;8(5):581-591.doi: 10.1177/1948550617693062. PMID:
Iyengar S, Greenhouse JB. Selection models and the file drawer problem. Statist Sci 1988;3(1):109-117.doi: 10.1214/ss/1177013012. PMID:
Silliman NP. Nonparametric classes of weight functions to model publicationbias. Biometrika 1997;84(4):909-918.doi. PMID:
McShane BB, Bockenholt U, Hansen KT. Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspect Psychol Sci 2016;11(5):730-749.doi: 10.1177/1745691616662243. PMID: 27694467
Rufibach K. Selection models with monotone weight functions in meta analysis. Biom J 2011;53(4):689-704.doi: 10.1002/bimj.201000240. PMID: 21567443
Citkowicz M, Vevea JL. A parsimonious weight function for modeling publication bias. Psychol Methods 2017;22(1):28-41.doi: 10.1037/met0000119. PMID: 28252998
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Jonny Karunia Fajar
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.