How can we address the surge of low-quality systematic reviews and their impact on high journal rejection rates?
Article Sidebar
Main Article Content
Journals have experienced a significant rise in submissions of systematic reviews and other types of reviews that often fall short of acceptable quality standards. These shortcomings typically stem from insufficient rigor in their methodology, reporting, or critical appraisal. As a result, these submissions are frequently rejected raising concerns about the standards authors are following when preparing such work. This growing trend of low-quality reviews not only places a burden on editorial teams but also poses a risk to the scientific community by potentially disseminating flawed or unreliable conclusions. Ensuring that articles maintain high standards is crucial for preserving the integrity of the scientific literature and facilitating evidence-based decision-making. In an effort to address this problem, this viewpoint editorial aims to offer concepts and recommendations on available tools for future authors to improve the quality of their reviews, as well as to guide readers and potential journal reviewers on how to critically interpret these articles.
- Systematic reviews
- methodological rigor
- journal rejection rates
- critical appraisal
- evidence-based tools
- Case-Control Studies
- Evidence-Based Practice
- Cross-Sectional Studies
- Expert Testimony
- Feasibility Studies
Marilina Santero, Universitat Autònoma Barcelona (UAB); Organización Mundial de la Salud (OMS); España
https://orcid.org/0000-0001-5371-0979
Universitat Autònoma Barcelona (UAB); Organización Mundial de la Salud (OMS); España
Samanta Díaz Menai, Universidad Hospital Italiano de Buenos Aires; Buenos Aires, Argentina.
Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and metaanalyses. Milbank Q. 2016;94: 485-514. https://doi.org/10.1111/1468-0009.12210 PMid:27620683 PMCid:PMC5020151 DOI: https://doi.org/10.1111/1468-0009.12210
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Syst Rev. 2023;12: 96. https://doi.org/10.2106/JBJS.RVW.23.00077 PMid:37285444 PMCid:PMC10259219 DOI: https://doi.org/10.1186/s13643-023-02255-9
Khadilkar SS. Rejection blues: Why do research papers get rejected? J Obstet Gynaecol India. 2018;68: 239-241. https://doi.org/10.1007/s13224-018-1153-1 PMid:30065536 PMCid:PMC6046667 DOI: https://doi.org/10.1007/s13224-018-1153-1
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009; 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x PMid:19490148 DOI: https://doi.org/10.1111/j.1471-1842.2009.00848.x
Tricco AC, Soobiah C, Antony J, Cogo E, MacDonald H, Lillie E, et al. A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method. J Clin Epidemiol. 2016;73: 19-28. https://doi.org/10.1016/j.jclinepi.2015.08.030 PMid:26891949 DOI: https://doi.org/10.1016/j.jclinepi.2015.08.030
Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z (editors). JBI Manual for Evidence Synthesis. JBI; 2024. doi: 10.46658/JBIMES-24-01. Available from: https://synthesismanual.jbi.global https://doi.org/10.46658/JBIMES-24-01 PMCid:PMC11342836 DOI: https://doi.org/10.46658/JBIMES-24-01
Cochrane. Interactive Learning. Accessed: august 25, 2025, Available from https://training.cochrane.org/ interactivelearning
Torraco RJ. Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review. 2005; 4(3): 356-367. https://doi.org/10.1177/1534484305278283 DOI: https://doi.org/10.1177/1534484305278283
Ehrich K, Freeman GK, Richards SC, Robinson IC, Shepperd S. How to do a scoping exercise: continuity of care. Research Policy Planning. 2002; 20: 25-29.
Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst. 2008;6: 7. https://doi.org/10.1186/1478-4505-6-7 PMid:18613961 PMCid:PMC2500008 DOI: https://doi.org/10.1186/1478-4505-6-7
Verdejo C, Tapia-Benavente L, Schuller-Martínez B, Vergara-Merino L, Vargas-Peirano M, Silva-Dreyer AM. Lo que tienes que saber sobre las revisiones panorámicas. Medwave. 2021; 21(2): e8144. https://doi.org/10.5867/medwave.2021.02.8144 PMid:33914717 DOI: https://doi.org/10.5867/medwave.2021.02.8144
Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169: 467-473. https://doi.org/10.7326/M18-0850 PMid:30178033 DOI: https://doi.org/10.7326/M18-0850
Belbasis L, Bellou V, Ioannidis JPA. Conducting umbrella reviews. BMJ Med. 2022; 1(1): e000071. https://doi.org/10.1136/bmjmed-2021-000071 PMid:36936579 PMCid:PMC9951359 DOI: https://doi.org/10.1136/bmjmed-2021-000071
Choi GJ, Kang H. Introduction to umbrella reviews as a useful evidence-based practice. J Lipid Atheroscler. 2023; 12(1): 3-11. doi: 10.12997/jla.2023.12.1.3
https://doi.org/10.12997/jla.2023.12.1.3 PMid:36761061 PMCid:PMC9884555 DOI: https://doi.org/10.12997/jla.2023.12.1.3
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al (editores). Manual Cochrane para revisiones sistemáticas de intervenciones, versión 6.5 (actualizado en agosto de 2024). Cochrane; 2024. Availabale from: https://www.training.cochrane.org/handbook
Turner T, Lavis JN, Grimshaw JM, Green S, Elliott J. Living evidence and adaptive policy: perfect partners? Health Res Policy Syst. 2023;21: 135. https://doi.org/10.1186/s12961-023-01085-4 PMid:38111030 PMCid:PMC10726516 DOI: https://doi.org/10.1186/s12961-023-01085-4
Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91: 23-30. https://doi.org/10.1016/j.jclinepi.2017.08.010 PMid:28912002 DOI: https://doi.org/10.1016/j.jclinepi.2017.08.010
Garritty C, Gartlehner G, Nussbaumer-Streit B, King VJ, Hamel C, Kamel C, et al. Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol. 2021;130: 1322. https://doi.org/10.1016/j.jclinepi.2020.10.007 PMid:33068715 PMCid:PMC7557165 DOI: https://doi.org/10.1016/j.jclinepi.2020.10.007
Escandón K, Rasmussen AL, Bogoch II, Murray EJ, Escandón K, Popescu SV, et al. COVID-19 false dichotomies and a comprehensive review of the evidence regarding public health, COVID-19 symptomatology, SARS-CoV-2 transmission, mask wearing, and reinfection. BMC Infect Dis. 2021;21: 710. https://doi.org/10.1186/s12879-021-06357-4 PMid:34315427 PMCid:PMC8314268 DOI: https://doi.org/10.1186/s12879-021-06357-4
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021; 372: n71. https://doi.org/10.1136/bmj.n71 PMid:33782057 PMCid:PMC8005924 DOI: https://doi.org/10.1136/bmj.n71
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358: j4008. https://doi.org/10.1136/bmj.j4008 PMid:28935701 PMCid:PMC5833365 DOI: https://doi.org/10.1136/bmj.j4008
Whiting P, Savovic J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69: 225-234. https://doi.org/10.1016/j.jclinepi.2015.06.005 PMid:26092286 PMCid:PMC4687950 DOI: https://doi.org/10.1016/j.jclinepi.2015.06.005
Amog K, Pham B, Courvoisier M, Mak M, Booth A, Godfrey C, et al. The web-based "Right Review" tool asks reviewers simple questions to suggest methods from 41 knowledge synthesis methods. J Clin Epidemiol. 2022;147: 42-51. https://doi.org/10.1016/j.jclinepi.2022.03.004 PMid:35314349 DOI: https://doi.org/10.1016/j.jclinepi.2022.03.004
Hosseini MS, Jahanshahlou F, Akbarzadeh MA, Zarei M, Vaez-Gharamaleki Y. Formulating research questions for evidence-based studies. Surgery Public Health. 2024;2. https://doi.org/10.1016/j.glmedi.2023.100046 DOI: https://doi.org/10.1016/j.glmedi.2023.100046
Willis LD. Formulating the research question and framing the hypothesis. Respir Care. 2023;68: 1180-1185. https://doi.org/10.4187/respcare.10975 PMid:37041024 PMCid:PMC10353175 DOI: https://doi.org/10.4187/respcare.10975
Doran GT. There's a SMART way to write managements's goals and objectives. Management Review. 1981; 70(11): 35-36.
Johnson BT, Hennessy EA. Systematic reviews and meta-analyses in the health sciences: Best practice methods for research syntheses. Soc Sci Med. 2019;233: 237-251. https://doi.org/10.1016/j.socscimed.2019.05.035 PMid:31233957 PMCid:PMC8594904 DOI: https://doi.org/10.1016/j.socscimed.2019.05.035
Haddaway NR, Grainger MJ, Gray CT. Citation chaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods. 2022;13: 533-545. https://doi.org/10.1002/jrsm.1563 PMid:35472127 DOI: https://doi.org/10.1002/jrsm.1563
Adam GP, Paynter R. Development of literature search strategies for evidence syntheses: pros and cons of incorporating text mining tools and objective approaches. BMJ Evid Based Med. 2023;28: 137-139. https://doi.org/10.1136/bmjebm-2021-111892 PMid:35346974 DOI: https://doi.org/10.1136/bmjebm-2021-111892
Pieper D, Rombey T. Where to prospectively register a systematic review. Syst Rev. 2022;11: 8. https://doi.org/10.1186/s13643-021-01877-1 PMid:34998432 PMCid:PMC8742923 DOI: https://doi.org/10.1186/s13643-021-01877-1
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4: 1. https://doi.org/10.1186/2046-4053-4-1 PMid:25554246 PMCid:PMC4320440 DOI: https://doi.org/10.1186/2046-4053-4-1
Sterne JAC, Savovic J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: l4898. https://doi.org/10.1136/bmj.l4898 PMid:31462531 DOI: https://doi.org/10.1136/bmj.l4898
Sterne JAC, Hernán MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016; i4919. https://doi.org/10.1136/bmj.i4919 PMid:27733354 PMCid:PMC5062054 DOI: https://doi.org/10.1136/bmj.i4919
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011; 64: 383-394. https://doi.org/10.1016/j.jclinepi.2010.04.026 PMid:21195583 DOI: https://doi.org/10.1016/j.jclinepi.2010.04.026
Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155: 529-536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009 PMid:22007046 DOI: https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education Information. 2018; 34(4): 285-291. https://doi.org/10.3233/EFI-180221 DOI: https://doi.org/10.3233/EFI-180221
Elite Providers Hub for Progressive Play. Quality assessment tool for quantitative studies; 2019. Cited 22 Nov 2024. Available: https://www.ephpp.ca/quality-assessment-tool-for-quantitative-studies/
Thomas H. Quality assessment tool for quantitative studies. Canada: Hamilton ON; 2003.
Review Manager (RevMan). Computer program. Version 8.9.0. The Cochrane Collaboration; 2024. Cited: august 25, 2025, Available from: https://revman.cochrane.org Review Manager (RevMan) Computer program. Version 8.9.0. The Cochrane Collaboration; 2024. [august 25, 2024]. Available from: https://revman.cochrane. org . [Google Scholar]
Comprehensive Meta-Analysis. Software (CMA.4) Computer program. Version 4. Cited: august 25, 2024, from: https://meta-analysis.com/ [august 25, 2024];Software (CMA.4) Computer program. Version 4. Cited: august 25, 2024, from: https://meta-analysis.com/
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews: A product from the ESRC Methods Programme. Lancaster, England: Lancaster University; 2006. doi:10.13140/2.1.1018.4643
Owen RK, Bradbury N, Xin Y, Cooper N, Sutton A. MetaInsight: An interactive web-based tool for analyzing, interrogating, and visualizing network meta-analyses using R-shiny and netmeta. Res Synth Methods. 2019;10: 569-581. https://doi.org/10.1002/jrsm.1373 PMid:31349391 PMCid:PMC6973101 DOI: https://doi.org/10.1002/jrsm.1373
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354: 1896-1900. https://doi.org/10.1016/S0140-6736(99)04149-5 PMid:10584742 DOI: https://doi.org/10.1016/S0140-6736(99)04149-5
Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000; 283: 2008-2012. https://doi.org/10.1001/jama.283.15.2008 PMid:10789670 DOI: https://doi.org/10.1001/jama.283.15.2008
Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336: 1106-1110. https://doi.org/10.1136/bmj.39500.677199.AE PMid:18483053 PMCid:PMC2386626 DOI: https://doi.org/10.1136/bmj.39500.677199.AE
GRADEpro GDT. Computer program. Hamilton (ON): McMaster University (developed by Evidence Prime), 2024. Cited: February 25, 2025, from https://www.gradepro.org/
Downloads
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The copy rights of the articles published in Colombia Médica belong to the Universidad del Valle. The contents of the articles that appear in the Journal are exclusively the responsibility of the authors and do not necessarily reflect the opinions of the Editorial Committee of the Journal. It is allowed to reproduce the material published in Colombia Médica without prior authorization for non-commercial use