Accurate Localization of 3D Objects from RGB-D Data using Segmentation Hypotheses

(Left) We propose a new framework to obtain accurate localizations of objects in 3D by exploring segmentation hypotheses of the object in 2D. (Right) The process of generating HFM and features from corresponding 3D point clouds. See details in the paper.

In this paper we focus on the problem of detecting objects in 3D from RGB-D images. We propose a novel framework that explores the compatibility between segmentation hypotheses of the object in the image and the corresponding 3D map. Our framework allows to discover the optimal location of the object using a generalization of the structural latent SVM formulation in 3D as well as the definition of a new loss function defined over the 3D space in training. We evaluate our method using two existing RGB-D datasets. Extensive quantitative and qualitative experimental results show that our proposed approach outperforms state-of-the-art as methods well as a number of baseline approaches for both 3D and 2D object recognition tasks.

Video

Publications

Byung-soo Kim, Shili Xu, and Silvio Savarese, "Accurate Localization of 3D Objects from RGB-D Data using Segmentation Hypotheses." CVPR'13 (pdf) (bibtex) (supplementary material will be updated soon.)

Code

Link

Acknowledgements

We acknowledge the support of ARO grant W911NF-09-1-0310, NSF CPS grant #0931474 and a KLA-Tencor Fellowship.

References

P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan "Object detection with discriminatively trained part-based models. " PAMI'10.
L. Bo, K. Lai, X. Ren, and D. Fox "Object recognition with hierarchical kernel descriptors. " ICCV'11.

Contact : bsookim at umich dot edu

Last update : July 10, 2013