Offspring from reproduction problems: What replication failure teaches us

Antske Fokkens, Marieke Van Erp, Marten Postma, Ted Pedersen, Piek Vossen, Nuno Freire

Research output: Chapter in Book/Report/Conference proceedingConference contribution

50 Scopus citations

Abstract

Repeating experiments is an important instrument in the scientific toolbox to validate previous work and build upon existing work. We present two concrete use cases involving key techniques in the NLP domain for which we show that reproducing results is still difficult. We show that the deviation that can be found in reproduction efforts leads to questions about how our results should be interpreted. Moreover, investigating these deviations provides new insights and a deeper understanding of the examined techniques. We identify five aspects that can influence the outcomes of experiments that are typically not addressed in research papers. Our use cases show that these aspects may change the answer to research questions leading us to conclude that more care should be taken in interpreting our results and more research involving systematic testing of methods is required in our field.

Original languageEnglish (US)
Title of host publicationLong Papers
PublisherAssociation for Computational Linguistics (ACL)
Pages1691-1701
Number of pages11
ISBN (Print)9781937284503
StatePublished - Jan 1 2013
Externally publishedYes
Event51st Annual Meeting of the Association for Computational Linguistics, ACL 2013 - Sofia, Bulgaria
Duration: Aug 4 2013Aug 9 2013

Publication series

NameACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
Volume1

Other

Other51st Annual Meeting of the Association for Computational Linguistics, ACL 2013
CountryBulgaria
CitySofia
Period8/4/138/9/13

Fingerprint Dive into the research topics of 'Offspring from reproduction problems: What replication failure teaches us'. Together they form a unique fingerprint.

Cite this