Assessing harmful effects in systematic reviews

Background: Balanced decisions about health care interventions require reliable evidence on harms as well as benefits. Most systematic reviews focus on efficacy and randomised trials, for which the methodology is well established. Methods to systematically review harmful effects are less well developed and there are few sources of guidance for researchers. We present our own recent experience of conducting systematic reviews of harmful effects and make suggestions for future practice and further research.

Methods: We described and compared the methods used in three systematic reviews. Our evaluation focused on the review question, study designs and quality assessment.

Results: One review question focused on providing information on specific harmful effects to furnish an economic model, the other two addressed much broader questions. All three reviews included randomised and observational data, although each defined the inclusion criteria differently. Standard methods were used to assess study quality. Various practical problems were encountered in applying the study design inclusion criteria and assessing quality, mainly because of poor study design, inadequate reporting and the limitations of existing tools. All three reviews generated a large volume of work that did not yield much useful information for health care decision makers. The key areas for improvement we identified were focusing the review question and developing methods for quality assessment of studies of harmful effects.

Conclusions: Systematic reviews of harmful effects are more likely to yield information pertinent to clinical decision-making if they address a focused question. This will enable clear decisions to be made about the type of research to include in the review. The methodology for assessing the quality of harmful effects data in systematic reviews requires further development.

Similar articles

Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.

Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834

Patnode CD, Henderson JT, Melnikow J, Coppola EL, Durbin S, Thomas R. Patnode CD, et al. Rockville (MD): Agency for Healthcare Research and Quality (US); 2021 Jan. Report No.: 20-05264-EF-1. Rockville (MD): Agency for Healthcare Research and Quality (US); 2021 Jan. Report No.: 20-05264-EF-1. PMID: 33523610 Free Books & Documents. Review.

Chen YF, Madan J, Welton N, Yahaya I, Aveyard P, Bauld L, Wang D, Fry-Smith A, Munafò MR. Chen YF, et al. Health Technol Assess. 2012;16(38):1-205, iii-v. doi: 10.3310/hta16380. Health Technol Assess. 2012. PMID: 23046909 Review.

Siskind DJ, Wu BT, Wong TT, Firth J, Kisely S. Siskind DJ, et al. Lancet Psychiatry. 2020 Sep;7(9):762-774. doi: 10.1016/S2215-0366(20)30261-3. Lancet Psychiatry. 2020. PMID: 32828166

Cited by

Qureshi R, Mayo-Wilson E, Rittiphairoj T, McAdams-DeMarco M, Guallar E, Li T. Qureshi R, et al. J Clin Epidemiol. 2022 Mar;143:212-223. doi: 10.1016/j.jclinepi.2021.10.024. Epub 2021 Nov 3. J Clin Epidemiol. 2022. PMID: 34742789 Free PMC article.

Rehman Y, Saini A, Huang S, Sood E, Gill R, Yanikomeroglu S. Rehman Y, et al. AIMS Neurosci. 2021 May 13;8(3):414-434. doi: 10.3934/Neuroscience.2021022. eCollection 2021. AIMS Neurosci. 2021. PMID: 34183989 Free PMC article. Review.

Li Y, Miao R, Liu Y, Zhang J, Dou Z, Zhao L, Zhang Y, Huang Z, Xia Y, Han D. Li Y, et al. Front Endocrinol (Lausanne). 2021 Apr 20;12:656621. doi: 10.3389/fendo.2021.656621. eCollection 2021. Front Endocrinol (Lausanne). 2021. PMID: 33959100 Free PMC article.

Montroy J, Berjawi R, Lalu MM, Podolsky E, Peixoto C, Sahin L, Stintzi A, Mack D, Fergusson DA. Montroy J, et al. BMC Gastroenterol. 2020 Nov 10;20(1):372. doi: 10.1186/s12876-020-01516-4. BMC Gastroenterol. 2020. PMID: 33167889 Free PMC article.

Smith SM, Dworkin RH, Turk DC, McDermott MP, Eccleston C, Farrar JT, Rowbotham MC, Bhagwagar Z, Burke LB, Cowan P, Ellenberg SS, Evans SR, Freeman RL, Garrison LP, Iyengar S, Jadad A, Jensen MP, Junor R, Kamp C, Katz NP, Kesslak JP, Kopecky EA, Lissin D, Markman JD, Mease PJ, O'Connor AB, Patel KV, Raja SN, Sampaio C, Schoenfeld D, Singh J, Steigerwald I, Strand V, Tive LA, Tobias J, Wasan AD, Wilson HD. Smith SM, et al. Pain. 2020 Nov;161(11):2446-2461. doi: 10.1097/j.pain.0000000000001952. Pain. 2020. PMID: 32520773 Free PMC article.

References

    1. Ernst E. Pittler MH Assessment of therapeutic safety in systematic reviews: literature review. BMJ. 2001;323:546. doi: 10.1136/bmj.323.7312.546. - DOI - PMC - PubMed
    1. Ioannidis JPA. Lau J Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA. 2001;285:437–443. - PubMed
    1. Loke YK. Derry S Reporting of adverse drug reactions in randomised controlled trials - a systematic survey. BMC Clin Pharmacol. 2001;1:3. doi: 10.1186/1472-6904-1-3. - DOI - PMC - PubMed
    1. Cuervo GL. Clarke M Balancing benefits and harms in health care. BMJ. 2003;327:65–66. doi: 10.1136/bmj.327.7406.65. - DOI - PMC - PubMed
    1. Aronson JK. Derry S. Loke YK Adverse drug reactions: keeping up to date. Fundam Clin Pharmacol. 2002;16:49–56. doi: 10.1046/j.1472-8206.2002.00066.x. - DOI - PubMed