Comparison of a web-push survey research protocol with a mailed paper and pencil protocol in the Monitoring the Future panel survey

Aims: The experiment tested the effects of a web-push survey research protocol, compared with the standard mailed paper-and-pencil protocol, among young adults aged 19–30 years in the ‘Monitoring the Future’ (MTF) longitudinal study. Design, Setting and Participants: The US-based MTF study has measured substance use trends among young adults in panel samples followed biennially, using consistent mailed survey procedures from 1977 to 2017. In 2018, young adult participants in the MTF longitudinal component scheduled to be surveyed at ages 19–30 in 2018 (from high school senior cohorts of 2006–17, n = 14 709) were randomly assigned to receive the standard mail/paper survey procedures or new web-push procedures. Measurements: Primary outcomes were responding to the survey and prevalence estimates for past 30-day use of alcohol, cigarettes, marijuana and illicit drugs. Findings: The web-push response rate was 39.07% [95% confidence interval (CI) = 37.889, 40.258]; this was significantly better than the standard MTF response rate of 35.12% (95% CI = 33.964, 36.285). After adjusting for covariates, the web-push condition was associated with a 19% increase in the odds of responding compared with standard MTF (adjusted odds ratio = 1.188; 95% CI = 1.096, 1.287). Substance use prevalence estimates were very similar and differences became negligible when using attrition weights and controlling for socio-demographic characteristics. Conclusions: The web-push protocol produced a higher response rate than the mailed pencil and paper protocol in the Monitoring the Future panel study, without substantially affecting estimates of substance use once attrition weights and socio-demographic variables were factored in.