Abstract
Widely-used public benchmarks are of huge importance to computer vision and machine learning research, especially with the computational resources required to reproduce state of the art results quickly becoming untenable. In medical image computing, the wide variety of image modalities and problem formulations yields a huge task-space for benchmarks to cover, and thus the widespread adoption of standard benchmarks has been slow, and barriers to releasing medical data exacerbate this issue. In this paper, we examine the role that publicly available data has played in MICCAI papers from the past five years. We find that more than half of these papers are based on private data alone, although this proportion seems to be decreasing over time. Additionally, we observed that after controlling for open access publication and the release of code, papers based on public data were cited over 60% more per year than their private-data counterparts. Further, we found that more than 20% of papers using public data did not provide a citation to the dataset or associated manuscript, highlighting the “second-rate” status that data contributions often take compared to theoretical ones. We conclude by making recommendations for MICCAI policies which could help to better incentivise data sharing and move the field toward more efficient and reproducible science.
| Original language | English (US) |
|---|---|
| Title of host publication | Large-Scale Annotation of Biomedical Data and Expert Label Synthesis and Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention - International Workshops, LABELS 2019, HAL-MICCAI 2019, and CuRIOUS 2019, held in Conjunction with MICCAI 2019, Proceedings |
| Editors | Luping Zhou, Nicholas Heller, Yiyu Shi, Danny Chen, X. Sharon Hu, Yiming Xiao, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Matthieu Chabanas, Hassan Rivaz, Ingerid Reinertsen |
| Publisher | Springer |
| Pages | 70-77 |
| Number of pages | 8 |
| ISBN (Print) | 9783030336417 |
| DOIs | |
| State | Published - 2019 |
| Event | 4th International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2019, the 1st International Workshop on Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention, HAL-MICCAI 2019, and the 2nd International Workshop on Correction of Brainshift with Intra-Operative Ultrasound, CuRIOUS 2019, held in conjunction with the 22nd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2019 - Shenzhen, China Duration: Oct 17 2019 → Oct 17 2019 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 11851 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | 4th International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2019, the 1st International Workshop on Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention, HAL-MICCAI 2019, and the 2nd International Workshop on Correction of Brainshift with Intra-Operative Ultrasound, CuRIOUS 2019, held in conjunction with the 22nd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2019 |
|---|---|
| Country/Territory | China |
| City | Shenzhen |
| Period | 10/17/19 → 10/17/19 |
Bibliographical note
Funding Information:Acknowledgements. Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA225435. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Publisher Copyright:
© Springer Nature Switzerland AG 2019.