Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists

Tatyana A. Shamliyan, Robert L Kane, Mohammed T. Ansari, Gowri Raman, Nancy D. Berkman, Mark Grant, Gail Janes, Margaret Maglione, David Moher, Mona Nasser, Karen A. Robinson, Jodi B. Segal, Sophia Tsouros

Research output: Contribution to journalArticlepeer-review

73 Scopus citations

Abstract

Objective: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

Original languageEnglish (US)
Pages (from-to)637-657
Number of pages21
JournalJournal of Clinical Epidemiology
Volume64
Issue number6
DOIs
StatePublished - Jun 2011

Bibliographical note

Funding Information:
Bush DE, Ziegelstein RC, Patel UV, Thombs BD, Ford DE, Fauerbach JA, et al. Post-Myocardial Infarction Depression. Evidence Report/Technology Assessment No. 123. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-02-0018.) AHRQ Publication No. 05-E018-2. Rockville, MD: Agency for Healthcare Research and Quality. May 2005.

Funding Information:
Buscemi N, Vandermeer B, Friesen C, Bialy L, Tubman M, Ospina M, et al. Manifestations and Management of Chronic Insomnia in Adults. Evidence Report/Technology Assessment No. 125. (Prepared by the University of Alberta Evidence-based Practice Center, under Contract No. C400000021.) AHRQ Publication No. 05-E021-2. Rockville, MD: Agency for Healthcare Research and Quality. June 2005.

Keywords

  • Bias (epidemiology)
  • Morbidity
  • Quality control
  • Reproducibility of results
  • Review literature as topic
  • Risk factors
  • Validation studies

Fingerprint Dive into the research topics of 'Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists'. Together they form a unique fingerprint.

Cite this