Measuring semantic traits for phenotyping is an essential but labor-intensive activity in horticulture. Researchers often rely on manual measurements which may not be accurate for tasks, such as measuring tree volume. To improve the accuracy of such measurements and to automate the process, we consider the problem of building coherent three-dimensional (3D) reconstructions of orchard rows. Even though 3D reconstructions of side views can be obtained using standard mapping techniques, merging the two side-views is difficult due to the lack of overlap between the two partial reconstructions. Our first main contribution in this paper is a novel method that utilizes global features and semantic information to obtain an initial solution aligning the two sides. Our mapping approach then refines the 3D model of the entire tree row by integrating semantic information common to both sides, and extracted using our novel robust detection and fitting algorithms. Next, we present a vision system to measure semantic traits from the optimized 3D model that is built from the RGB or RGB-D data captured by only a camera. Specifically, we show how canopy volume, trunk diameter, tree height, and fruit count (FC) can be automatically obtained in real orchard environments. The experiment results from multiple data sets quantitatively demonstrate the high accuracy and robustness of our method.
Bibliographical noteFunding Information:
We thank Professors James Luby, Cindy Tong, and Emily Hoover from the Department of Horticultural Science, University of Minnesota, for their expertize and help with the experiments. We also thank our colleagues Joshua Anderson, Cheng Peng, and Nicolai Häni from the University of Minnesota, for providing valuable feedback throughout this study. Finally, we thank the anonymous reviewers for their careful reading of our manuscript and their insightful comments and suggestions. This study is supported in part by USDA NIFA MIN‐98‐G02, and a subgrant from NSF (# 1722310). https://github.com/matterport/Mask_RCNN http://www.robots.ox.ac.uk/~vgg/software/via