Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence

M. Hassan Murad; Stephanie M. Chang; Celia V. Fiordalisi; Jennifer S. Lin; Timothy J. Wilt; Amy Y. Tsou; Brian Leas; Shazia Siddique; Andrew R. Zullo; Ethan M. Balk; Carolyn M. Rutter; Karen Robinson; Craig Coleman; Olivia Costa; Mark Helfand; Elizabeth Stoeger; Meera Viswanathan

doi:10.1016/j.jclinepi.2021.02.028

Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence

M. Hassan Murad, Stephanie M. Chang, Celia V. Fiordalisi, Jennifer S. Lin, Timothy J. Wilt, Amy Y. Tsou, Brian Leas, Shazia Siddique, Andrew R. Zullo, Ethan M. Balk, Carolyn M. Rutter, Karen Robinson, Craig Coleman, Olivia Costa, Mark Helfand, Elizabeth Stoeger, Meera Viswanathan

Medicine - Veteran's Administration Medical Center

Research output: Contribution to journal › Article › peer-review

6 Scopus citations

Abstract

Objective: To identify and suggest strategies to make insufficient evidence ratings in systematic reviews more actionable. Study Design and Setting: A workgroup comprising members from the Evidence-Based Practice (EPC) Program of the Agency for Healthcare Research and Quality convened throughout 2020. We conducted iterative discussions considering information from three data sources: a literature review for relevant publications and frameworks, a review of a convenience sample of past systematic reviews conducted by the EPCs, and an audit of methods used in past EPC technical briefs. Results: We identified five strategies for supplementing systematic review findings when evidence on benefits or harms is expected to be, or found to be, insufficient: 1) reconsider eligible study designs, 2) summarize indirect evidence, 3) summarize contextual and implementation evidence, 4) consider modelling, and 5) incorporate unpublished health system data in the evidence synthesis. While these strategies may not increase the strength of evidence, they may improve the utility of reports for decision makers. Adopting these strategies depends on feasibility, timeline, funding, and expertise of the systematic reviewers. Conclusion: Throughout the process of evidence synthesis of early scoping, protocol development, review conduct, and review presentation, authors can consider these five strategies to supplement evidence with insufficient rating to make it more actionable for end-users.

Original language	English (US)
Pages (from-to)	170-175
Number of pages	6
Journal	Journal of Clinical Epidemiology
Volume	135
DOIs	https://doi.org/10.1016/j.jclinepi.2021.02.028
State	Published - Jul 2021

Bibliographical note

Funding Information:
Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center (290-2017-00003-C), Mayo Clinic EPC (290-2015-00013-I), University of Minnesota EPC (290-2015-00008-I), Kaiser Permanente Research Affiliates EPC (290-2015-00007-I), RTI-University of North Carolina EPC (290-2015-00011-I), Southern California/RAND EPC (290-2015-00010-I), Brown University EPC (290-2015-00002-I), ECRI Institute-Penn Medicine EPC (290-2015-00005-I), University of Connecticut EPC (290-2015-00012-I), and Johns Hopkins University EPC (290-2015-00006-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

Funding Information:
Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center ( 290-2017-00003-C ), Mayo Clinic EPC ( 290-2015-00013-I ), University of Minnesota EPC ( 290-2015-00008-I ), Kaiser Permanente Research Affiliates EPC ( 290-2015-00007-I ), RTI- University of North Carolina EPC ( 290-2015-00011-I ), Southern California / RAND EPC ( 290-2015-00010-I ), Brown University EPC ( 290-2015-00002-I ), ECRI Institute-Penn Medicine EPC ( 290-2015-00005-I ), University of Connecticut EPC ( 290-2015-00012-I ), and Johns Hopkins University EPC ( 290-2015-00006-I ). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

Publisher Copyright:
© 2021 The Authors

Keywords

Decision-making
Health systems
Insufficient evidence
Systematic reviews
Translation

Access

10.1016/j.jclinepi.2021.02.028

OpenUrl availability

Full text

Cite this

Murad, M. H., Chang, S. M., Fiordalisi, C. V., Lin, J. S., Wilt, T. J., Tsou, A. Y., Leas, B., Siddique, S., Zullo, A. R., Balk, E. M., Rutter, C. M., Robinson, K., Coleman, C., Costa, O., Helfand, M., Stoeger, E., & Viswanathan, M. (2021). Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence. Journal of Clinical Epidemiology, 135, 170-175. https://doi.org/10.1016/j.jclinepi.2021.02.028

Murad, MH, Chang, SM, Fiordalisi, CV, Lin, JS, Wilt, TJ, Tsou, AY, Leas, B, Siddique, S, Zullo, AR, Balk, EM, Rutter, CM, Robinson, K, Coleman, C, Costa, O, Helfand, M, Stoeger, E & Viswanathan, M 2021, 'Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence', Journal of Clinical Epidemiology, vol. 135, pp. 170-175. https://doi.org/10.1016/j.jclinepi.2021.02.028

@article{7e4d131c145d48ea99f65fb473721071,

title = "Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence",

abstract = "Objective: To identify and suggest strategies to make insufficient evidence ratings in systematic reviews more actionable. Study Design and Setting: A workgroup comprising members from the Evidence-Based Practice (EPC) Program of the Agency for Healthcare Research and Quality convened throughout 2020. We conducted iterative discussions considering information from three data sources: a literature review for relevant publications and frameworks, a review of a convenience sample of past systematic reviews conducted by the EPCs, and an audit of methods used in past EPC technical briefs. Results: We identified five strategies for supplementing systematic review findings when evidence on benefits or harms is expected to be, or found to be, insufficient: 1) reconsider eligible study designs, 2) summarize indirect evidence, 3) summarize contextual and implementation evidence, 4) consider modelling, and 5) incorporate unpublished health system data in the evidence synthesis. While these strategies may not increase the strength of evidence, they may improve the utility of reports for decision makers. Adopting these strategies depends on feasibility, timeline, funding, and expertise of the systematic reviewers. Conclusion: Throughout the process of evidence synthesis of early scoping, protocol development, review conduct, and review presentation, authors can consider these five strategies to supplement evidence with insufficient rating to make it more actionable for end-users.",

keywords = "Decision-making, Health systems, Insufficient evidence, Systematic reviews, Translation",

author = "Murad, {M. Hassan} and Chang, {Stephanie M.} and Fiordalisi, {Celia V.} and Lin, {Jennifer S.} and Wilt, {Timothy J.} and Tsou, {Amy Y.} and Brian Leas and Shazia Siddique and Zullo, {Andrew R.} and Balk, {Ethan M.} and Rutter, {Carolyn M.} and Karen Robinson and Craig Coleman and Olivia Costa and Mark Helfand and Elizabeth Stoeger and Meera Viswanathan",

note = "Funding Information: Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center (290-2017-00003-C), Mayo Clinic EPC (290-2015-00013-I), University of Minnesota EPC (290-2015-00008-I), Kaiser Permanente Research Affiliates EPC (290-2015-00007-I), RTI-University of North Carolina EPC (290-2015-00011-I), Southern California/RAND EPC (290-2015-00010-I), Brown University EPC (290-2015-00002-I), ECRI Institute-Penn Medicine EPC (290-2015-00005-I), University of Connecticut EPC (290-2015-00012-I), and Johns Hopkins University EPC (290-2015-00006-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. Funding Information: Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center ( 290-2017-00003-C ), Mayo Clinic EPC ( 290-2015-00013-I ), University of Minnesota EPC ( 290-2015-00008-I ), Kaiser Permanente Research Affiliates EPC ( 290-2015-00007-I ), RTI- University of North Carolina EPC ( 290-2015-00011-I ), Southern California / RAND EPC ( 290-2015-00010-I ), Brown University EPC ( 290-2015-00002-I ), ECRI Institute-Penn Medicine EPC ( 290-2015-00005-I ), University of Connecticut EPC ( 290-2015-00012-I ), and Johns Hopkins University EPC ( 290-2015-00006-I ). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. Publisher Copyright: {\textcopyright} 2021 The Authors",

year = "2021",

month = jul,

doi = "10.1016/j.jclinepi.2021.02.028",

language = "English (US)",

volume = "135",

pages = "170--175",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier USA",

}

TY - JOUR

T1 - Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence

AU - Murad, M. Hassan

AU - Chang, Stephanie M.

AU - Fiordalisi, Celia V.

AU - Lin, Jennifer S.

AU - Wilt, Timothy J.

AU - Tsou, Amy Y.

AU - Leas, Brian

AU - Siddique, Shazia

AU - Zullo, Andrew R.

AU - Balk, Ethan M.

AU - Rutter, Carolyn M.

AU - Robinson, Karen

AU - Coleman, Craig

AU - Costa, Olivia

AU - Helfand, Mark

AU - Stoeger, Elizabeth

AU - Viswanathan, Meera

N1 - Funding Information: Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center (290-2017-00003-C), Mayo Clinic EPC (290-2015-00013-I), University of Minnesota EPC (290-2015-00008-I), Kaiser Permanente Research Affiliates EPC (290-2015-00007-I), RTI-University of North Carolina EPC (290-2015-00011-I), Southern California/RAND EPC (290-2015-00010-I), Brown University EPC (290-2015-00002-I), ECRI Institute-Penn Medicine EPC (290-2015-00005-I), University of Connecticut EPC (290-2015-00012-I), and Johns Hopkins University EPC (290-2015-00006-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. Funding Information: Funding: This work was funded by the Agency for Healthcare Research and Quality through the following contracts: Scientific Resource Center ( 290-2017-00003-C ), Mayo Clinic EPC ( 290-2015-00013-I ), University of Minnesota EPC ( 290-2015-00008-I ), Kaiser Permanente Research Affiliates EPC ( 290-2015-00007-I ), RTI- University of North Carolina EPC ( 290-2015-00011-I ), Southern California / RAND EPC ( 290-2015-00010-I ), Brown University EPC ( 290-2015-00002-I ), ECRI Institute-Penn Medicine EPC ( 290-2015-00005-I ), University of Connecticut EPC ( 290-2015-00012-I ), and Johns Hopkins University EPC ( 290-2015-00006-I ). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. Publisher Copyright: © 2021 The Authors

PY - 2021/7

Y1 - 2021/7

N2 - Objective: To identify and suggest strategies to make insufficient evidence ratings in systematic reviews more actionable. Study Design and Setting: A workgroup comprising members from the Evidence-Based Practice (EPC) Program of the Agency for Healthcare Research and Quality convened throughout 2020. We conducted iterative discussions considering information from three data sources: a literature review for relevant publications and frameworks, a review of a convenience sample of past systematic reviews conducted by the EPCs, and an audit of methods used in past EPC technical briefs. Results: We identified five strategies for supplementing systematic review findings when evidence on benefits or harms is expected to be, or found to be, insufficient: 1) reconsider eligible study designs, 2) summarize indirect evidence, 3) summarize contextual and implementation evidence, 4) consider modelling, and 5) incorporate unpublished health system data in the evidence synthesis. While these strategies may not increase the strength of evidence, they may improve the utility of reports for decision makers. Adopting these strategies depends on feasibility, timeline, funding, and expertise of the systematic reviewers. Conclusion: Throughout the process of evidence synthesis of early scoping, protocol development, review conduct, and review presentation, authors can consider these five strategies to supplement evidence with insufficient rating to make it more actionable for end-users.

AB - Objective: To identify and suggest strategies to make insufficient evidence ratings in systematic reviews more actionable. Study Design and Setting: A workgroup comprising members from the Evidence-Based Practice (EPC) Program of the Agency for Healthcare Research and Quality convened throughout 2020. We conducted iterative discussions considering information from three data sources: a literature review for relevant publications and frameworks, a review of a convenience sample of past systematic reviews conducted by the EPCs, and an audit of methods used in past EPC technical briefs. Results: We identified five strategies for supplementing systematic review findings when evidence on benefits or harms is expected to be, or found to be, insufficient: 1) reconsider eligible study designs, 2) summarize indirect evidence, 3) summarize contextual and implementation evidence, 4) consider modelling, and 5) incorporate unpublished health system data in the evidence synthesis. While these strategies may not increase the strength of evidence, they may improve the utility of reports for decision makers. Adopting these strategies depends on feasibility, timeline, funding, and expertise of the systematic reviewers. Conclusion: Throughout the process of evidence synthesis of early scoping, protocol development, review conduct, and review presentation, authors can consider these five strategies to supplement evidence with insufficient rating to make it more actionable for end-users.

KW - Decision-making

KW - Health systems

KW - Insufficient evidence

KW - Systematic reviews

KW - Translation

UR - http://www.scopus.com/inward/record.url?scp=85103928066&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85103928066&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2021.02.028

DO - 10.1016/j.jclinepi.2021.02.028

M3 - Article

C2 - 33753229

AN - SCOPUS:85103928066

SN - 0895-4356

VL - 135

SP - 170

EP - 175

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Improving the utility of evidence synthesis for decision makers in the face of insufficient evidence

Abstract

Bibliographical note

Keywords

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this