Evaluating forecasts of political conflict dynamics

Patrick T. Brandt, John R. Freeman, Philip A. Schrodt

Research output: Contribution to journalArticlepeer-review

20 Scopus citations

Abstract

There is considerable interest today in the forecasting of conflict dynamics. Commonly, the root mean square error and other point metrics are used to evaluate the forecasts from such models. However, conflict processes are non-linear, so these point metrics often do not produce adequate evaluations of the calibration and sharpness of the forecast models. Forecast density evaluation improves the model evaluation. We review tools for density evaluation, including continuous rank probability scores, verification rank histograms, and sharpness plots. The usefulness of these tools for evaluating conflict forecasting models is explained. We illustrate this, first, in a comparison of several time series models' forecasts of simulated data from a Markov-switching process, and second, in a comparison of several models' abilities to forecast conflict dynamics in the Cross Straits. These applications show the pitfalls of relying on point metrics alone for evaluating the quality of conflict forecasting models. As in other fields, it is more useful to employ a suite of tools. A non-linear vector autoregressive model emerges as the model which is best able to forecast conflict dynamics between China and Taiwan.

Original languageEnglish (US)
Pages (from-to)944-962
Number of pages19
JournalInternational Journal of Forecasting
Volume30
Issue number4
DOIs
StatePublished - Oct 2014

Bibliographical note

Funding Information:
Earlier versions of this paper were presented at the Annual Meeting of the Midwest Political Science Association, Chicago, April 2012, and at the summer meeting of the Society for Political Methodology, Princeton University, July 2011. Drafts of this paper were also presented at colloquia at Princeton University, Texas A & M University, University of Pittsburgh, and University of Rochester. We thank the members of the Society for Political Methodology, participants at the colloquia, and especially Robert Erikson, Eleonora Mattiacci, Xun Pang, and Michael D. Ward for comments. This research is supported by the U.S. National Science Foundation , award numbers SES-0921051 , SES-0921018 , and SES-1004414 . The authors are responsible for the contents.

Funding Information:
Phillip A. Schrodt is Professor of Political Science at Pennsylvania State University. Prior to joining Penn State in 2010, he taught for 21 years at the University of Kansas, and 11 years at Northwestern University. His areas of research are quantitative models of political conflict and computational methodology. His current research focuses on predicting political change using statistical and pattern recognition methods, and has been supported by the National Science Foundation, the Defense Advanced Research Projects Agency, and the U.S. government’s Political Instability Task Force. He is a past president and current fellow of the Society for Political Methodology.

Copyright:
Copyright 2014 Elsevier B.V., All rights reserved.

Keywords

  • Bayesian
  • Conflict dynamics
  • Density evaluation
  • Scoring rules
  • Time series
  • Verification rank histogram

Fingerprint Dive into the research topics of 'Evaluating forecasts of political conflict dynamics'. Together they form a unique fingerprint.

Cite this