Title:

Extending 3D Reconstruction to Temporal and Multi-model Sensor Data for Precision Agriculture

Jing Dong

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: Thursday, August 23, 2018

Time: 1 - 3pm (EDT)

Location: TSRB 223

Committee:

Dr. Frank Dellaert (Advisor, School of Interactive Computing, Georgia Institute of Technology)

Dr. Byron Boots (Co-Advisor, School of Interactive Computing, Georgia Institute of Technology)

Dr. James Rehg (School of Interactive Computing, Georgia Institute of Technology)

Dr. Patricio Vela (School of Electrical and Computer Engineering, Georgia Institute of Technology)

Dr. Sudipta Sinha (Microsoft Research)

Abstract:

Crop monitoring is one of the most important tasks in precision agriculture, and to reduce cost, such task is often performed autonomously by unmanned aerial and ground vehicles. To capture 3D geometric information about crops, existing systems mostly use LIDAR, but LIDAR is expensive and there is a desire to replace it with cheaper sensors like monocular cameras coupled with techniques to obtain 3D reconstructions from 2D images. One of the major disadvantages of many existing 3D reconstruction algorithms is that they assume the scene is static, and they cannot be used to monitor crops growing over time. Moreover, many existing 3D reconstruction algorithms are not designed to handle multi-spectral or hyper-spectral images, which are commonly used in precision agriculture to recover information that cannot be seen by naked eye. 

In this work I propose a full pipeline for building 3D reconstructions from temporal and multi-modal image sequences to use in precision agriculture applications. The three main technical contributions are: (1) 3D reconstruction for low-cost systems enabled by Gaussian process based continuous-time SLAM, (2) spatio-temporal 4D reconstruction to enable the monitoring of crops over time, and (3) weakly-supervised learning of local image descriptors between multiple image modalities. We also collected, in collaboration with University of Georgia, a multi-year growing crop dataset at Tifton, GA, in order to evaluate the performance of the proposed pipeline.