Exercise tracking apps are a novel, scalable, and affordable tool for delivering personalized behavioral interventions. While thousands of fitness tracking solutions emerge in the market, there is a lack of systematic research that quantify their effectiveness on exercise outcomes, making it hard for for practitioners and users to know the true value of these apps. Drawing on the literature on motivation literature, this paper elucidates the effects of app-enabled motivation on running performance. Specifically, this study examines the two most common forms of feedback available to users of exercise tracking apps, namely performance feedback and social feedback. We conducted a 18-month long randomized field experiment, with 1,241 military servicemen, to assess the causal effect of these feedback types on actual exercise outcomes. Results from the experiment provided evidence that these two app features improve the running times of the servicemen. We further discuss the temporality and hetoerogeneity of these effects.