Semantic mapping for orchard environments by merging two-sides reconstructions of tree rows

Wenbo Dong, Pravakar Roy, Volkan Isler

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Measuring semantic traits for phenotyping is an essential but labor-intensive activity in horticulture. Researchers often rely on manual measurements which may not be accurate for tasks, such as measuring tree volume. To improve the accuracy of such measurements and to automate the process, we consider the problem of building coherent three-dimensional (3D) reconstructions of orchard rows. Even though 3D reconstructions of side views can be obtained using standard mapping techniques, merging the two side-views is difficult due to the lack of overlap between the two partial reconstructions. Our first main contribution in this paper is a novel method that utilizes global features and semantic information to obtain an initial solution aligning the two sides. Our mapping approach then refines the 3D model of the entire tree row by integrating semantic information common to both sides, and extracted using our novel robust detection and fitting algorithms. Next, we present a vision system to measure semantic traits from the optimized 3D model that is built from the RGB or RGB-D data captured by only a camera. Specifically, we show how canopy volume, trunk diameter, tree height, and fruit count (FC) can be automatically obtained in real orchard environments. The experiment results from multiple data sets quantitatively demonstrate the high accuracy and robustness of our method.

Original languageEnglish (US)
Pages (from-to)97-121
Number of pages25
JournalJournal of Field Robotics
Volume37
Issue number1
DOIs
StatePublished - Jan 1 2020

Bibliographical note

Funding Information:
We thank Professors James Luby, Cindy Tong, and Emily Hoover from the Department of Horticultural Science, University of Minnesota, for their expertize and help with the experiments. We also thank our colleagues Joshua Anderson, Cheng Peng, and Nicolai Häni from the University of Minnesota, for providing valuable feedback throughout this study. Finally, we thank the anonymous reviewers for their careful reading of our manuscript and their insightful comments and suggestions. This study is supported in part by USDA NIFA MIN‐98‐G02, and a subgrant from NSF (# 1722310). https://github.com/matterport/Mask_RCNN http://www.robots.ox.ac.uk/~vgg/software/via

Keywords

  • agriculture
  • mapping
  • perception

Fingerprint Dive into the research topics of 'Semantic mapping for orchard environments by merging two-sides reconstructions of tree rows'. Together they form a unique fingerprint.

Cite this