Research on 3D Reconstruction Methods Based on Deep Learning
Deep learning applications have been applied extensively and have made tremendous strides in the 3D reconstruction field in recent years. This paper offers a methodical review of deep learning-based techniques for single-view images, multi-view images, and video-based sequence approaches. For single-view methods, we focus on depth estimation using Convolutional Neural Networks (CNNs) and image-to-depth mapping using Generative Adversarial Networks (GANs). For multi-view methods, we explore 3D reconstruction based on multi-view stereo matching method, 3D points cloud reconstruction method and stereo flow estimation method. For video-based methods, we introduce depth estimation method based on optical flow and video sequence modeling using Recurrent Neural Networks (RNNs). Generally, the multi-view image method is more accurate and sophisticated than the single-view image method, while the method based on video sequences is more challenging and complex. Different 3D reconstruction methods depend on specific application scenarios and requirements. This review provides a considerable insight in research for 3D reconstruction and also make the conclusion as well as future prospect for this field.
References
- Bhoi, A. Monocular depth estimation: A survey. arXiv preprint arXiv:1901.09402. 2019
- Ming, Y., Meng, X., Fan, C., & Yu, H. (2021). Deep learning for monocular depth estimation: A review. Neurocomputing, 438, 14-33.
- Aleotti, F., Tosi, F., Poggi, M., & Mattoccia, S. (2018). Generative adversarial networks for unsupervised monocular depth prediction. ECCV workshops. 2018.
- Roca-Pardinas, J., Lorenzo, H., Arias, P., & Armesto, J. From laser point clouds to surfaces: Statistical nonparametric methods for three-dimensional reconstruction. Computer-Aided Design, 40(5), 646-652, 2008
- Iglhaut, J., Cabo, C., Puliti, S., Piermattei, L., O’Connor, J., & Rosette, J. Structure from motion photogrammetry in forestry: A review. Current Forestry Reports, 5, 155-168, 2019.
- Zhou, Y., Zhang, L., Xing, C., Xie, P., & Cao, Y. Target three-dimensional reconstruction from the multi-view radar image sequence. IEEE Access, 7, 36722-36735, 2019
- Luo, X., Huang, J. B., Szeliski, R., Matzen, K., & Kopf, J. Consistent video depth estimation. ACM ToG, 39(4), 71-1, 2020
- Wang, Y., Wang, P., Yang, Z., Luo, C., Yang, Y., & Xu, W. Unos: Unified unsupervised optical-flow and stereo-depth estimation by watching videos. In CVPR (pp. 8071-8081, 2019
- Lu, W., Cui, J., Chang, Y., & Zhang, L. A video prediction method based on optical flow estimation and pixel generation. IEEE Access, 9, 100395-100406, 2021
- Teed, Z., & Deng, J. Deepv2d: Video to depth with differentiable structure from motion. arXiv preprint arXiv:1812.04605, 2018.