Title:  Joint 3D Object Detection and Re-identification for Dynamic Indoor Scene Understanding

Committee: 

Dr. Issa, Advisor

Dr. Romberg, Co-Advisor     

Dr. Al-Regib, Chair

Dr. Batra

Abstract: The object of the proposed research is to address the tasks of 3D object detection and multi-class object re-identification jointly in indoor environments. We consider the case of an embodied agent exploring a house at two different times. In between the two explorations, the scene layout may have changed: objects may have been moved, added or removed from the original stage. The agent is equipped with a localized RGB-D camera and explores an environment following a given navigation trajectory. The objective is to re-identify objects detected during the second tour with objects detected the first time. We formulate the following hypothesis: Information gathered during the first exploration can be re-used to enhance the detection and matching performances of the second exploration. We propose a joint solution to this problem by leveraging contextual features from the first scene and use it for both detection and re-identification in the second scene. The final architecture will extend the VoteNet 3D object detector [1] and SuperGlue matching module [2] in a combined architecture. We will construct a new dataset of pairs of environments with different layout settings along with ground truth object matches. We will use it to validate our hypothesis by comparing performances of our proposed approach toa tracking-by-detection baseline where detection and matching are performed separately.