Online question and answer (Q&A) sites, which are platforms for users to post and answer questions on a wide range of topics, are becoming large repositories of valuable knowledge and important to societies. In order to sustain success, Q&A sites face the challenges of ensuring content quality and encouraging user contributions. This paper examines a particular design decision in Q&A sites-allowing Wikipedia-like collaborative editing on questions and answers, and explores its beneficial effects on content quality and potential detrimental effects on users' contributions. By examining five years' archival data of Stack Overflow, we found that the benefits of collaborative editing outweigh its risks. For example, each substantive edit from other users can increase the number of positive votes by 181% for the questions and 119% for the answers. On the other hand, each edit only decreases askers and answerers' subsequent contributions by no more than 5%. This work has implications for understanding and designing large-scale social computing systems.