Dr. Zhang now is a full professor at Zhejiang University. He received the B.S. and Ph.D. degree in computer science and technology from Zhejiang University in 2003 and 2009, respectively. He is a core member of Computer Vision Group at State Key Lab of CAD&CG, Zhejiang University.
Research Interests:
SfM/SLAM & 3D Reconstruction
Video Enhancement & Editing
Augmented Reality
News:
July 2018: We have released the source code of ENFT-SfM, SegmentBA, EIBA and ICE-BA. Please visit https://github.com/zju3dv/ for more details.
RKSLAM: Robust Keyframe-based Monocular SLAM for AR
RKSLAM is a real-time monocular simultaneous localization and mapping system which can robustly work in challenging cases, such as fast motion and strong rotation. It can run real-time on a mobile device and outperform state-of-the-art systems (e.g. ORB-SLAM, PTAM, LSD-SLAM) in challenging cases of fast motion and strong rotation.
LS-ACTS: Large-Scale Automatic Camera Tracking System
LS-ACTS is a robust and efficient structure-from-motion system which can recover camera motion and 3D scene structure from large videos/sequences datasets. Compared to our previous SfM system ACTS, it is much faster (near real-time in a normal desktop PC) and can handle multiple extremely long sequences (over 100K frames).
RDSLAM: Robust Dynamic Simultaneous Localization and Mapping
RDSLAMis a real-time simultaneous localization and mapping system which allows parts of the scene to be dynamic or the whole scene to gradually change. Compared to PTAM, RDSLAM not only can robustly work in dynamic environments, but also can handle a larger scale scene (the number of the reconstructed 3D points can be tens of thousands). It is the basis for many applications, such as real-time 3D reconstruction and augmented reality.
ACTS is an automatic camera tracking system which can recover camera motion and 3D scene structure from videos and film sequences, providing the ease of automatic tracking. It can track all kinds of the camera motion efficiently and stably, which can be rotational or free-moving. Especially, the long sequences with varying focal length can be handled in a robust way. Besides camera motion, it also can recover accurate and dense depth maps now.
We will actively update the program and add more and more advanced or extra functions along with our published papers in the future.
Publications:
† indicates joint first authors.
* indicates corresponding author.
Technical Reports:
Haomin Liu, Chen Li, Guojun Chen, Guofeng Zhang*, Michael Kaess and Hujun Bao. Robust Keyframe-based Dense SLAM with an RGB-D Camera. arXiv preprint arXiv:1711.05166, 2017. [arXiv report][source code]
Guofeng Zhang, Haomin Liu, Zilong Dong, Jiaya Jia, Tien-Tsin Wong and Hujun Bao. ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion. Technical Report (arXiv:1510.08012), October, 2015. [arXiv report][video][software]
Hai Li†, Hongjia Zhai†, Xingrui Yang, Zhirong Wu, Yihao Zheng, Haofan Wang, Jianchao Wu*, Hujun Bao, and Guofeng Zhang*.
ImTooth: Neural Implicit Tooth for Dental Augmented Reality.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2023.
[pdf]
[Presentation Video]
[Demo Video]
Hai Li†, Xingrui Yang†, Hongjia Zhai, Yuqian Liu, Hujun Bao, and Guofeng Zhang*.
Vox-Surf: Voxel-based Implicit Surface Representation.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022.
[pdf]
[source code]
Zhaoyang Huang, Xiaokun Pan, Weihong Pan, Weikang Bian, Yan Xu, Ka Chun Cheung, Guofeng Zhang*, Hongsheng Li*.
NeuralMarker: A Framework for Learning General Marker Correspondence.
ACM Transactions on Graphics (SIGGRAPH Asia), 2022.
[pdf]
[source code]
[project]
[video]
Kangkan Wang, Sida Peng, Xiaowei Zhou, Jian Yang, and Guofeng Zhang*.
NerfCap: Human Performance Capture with Dynamic Neural Radiance Fields.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022.
[pdf]
[video]
Zhichao Ye, Guanglin Li, Haomin Liu, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang*.
CoLi-BA: Compact Linearization based Solver for Bundle Adjustment.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022.
[pdf]
[video]
[source code]
Bangbang Yang, Yinda Zhang, Yijin Li, Zhaopeng Cui*, Sean Fanello, Hujun Bao, Guofeng Zhang*.
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects.
ACM Transactions on Graphics (SIGGRAPH), 2022.
[pdf]
[pdf-high-res]
[source code]
[project]
[video]
Hujun Bao, Weijian Xie, Quanhao Qian, Danpeng Cheng, Shangjin Zhai, Nan Wang, Guofeng Zhang*.
Robust Tightly-Coupled Visual-Inertial Odometry with Pre-built Maps in High Latency Situations.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 28(5): 2212-2222, 2022.
[pdf]
[video_presentation]
[video_experiment]
Guofeng Zhang, Xiaowei Zhou, Feng Tian, Hongbin Zha, Yongtian Wang, Hujun Bao.
The Present and Future of Mixed Reality in China.
Communications of the ACM (CACM), November 2021, Vol. 64 No. 11, Pages 64-69.
[pdf]
Xiaojun Xiang, Hanqing Jiang, Guofeng Zhang, Yihao Yu, Chenchen Li, Xingbin Yang, Danpeng Chen, Hujun Bao*.
Mobile3DScanner: An Online 3D Scanner for High-quality Object Reconstruction with a Mobile Device.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021 (ISMAR Best Journal Paper Nominee).
[pdf]
[project page]
[video-1]
[video-2]
Linghao Chen, Jiaming Sun, Yiming Xie, Siyu Zhang, Qing Shuai, Qinhong Jiang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou. Shape Prior Guided Instance Disparity Estimation for 3D Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. [pdf][video][source code]
Kangkan Wang*, Guofeng Zhang, Huayu Zheng, Jian Yang.
Learning Dense Correspondences for Non-Rigid Point Clouds With Two-Stage Regression.
IEEE Transactions on Image Processing (TIP), 30: 8468-8482, 2021.
[pdf]
Jundan Luo, Zhaoyang Huang, Yijin Li, Xiaowei Zhou, Guofeng Zhang, Hujun Bao*.
NIID-Net: Adapting Surface Normal Knowledge for Intrinsic Image Decomposition in Indoor Scenes.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 26(12): 3434-3445, 2020.
[pdf]
[supplementary document]
[video]
[source code]
Xingbin Yang, Liyang Zhou, Hanqing Jiang, Zhongliang Tang, Yuanbo Wang, Hujun Bao, Guofeng
Zhang*.
Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 26(12): 3446-3556, 2020 (ISMAR Best Paper Award).
[pdf]
[project page]
[video-1]
[video-2]
Jinyu Li, Bangbang Yang, Danpeng Chen, Nan Wang, Guofeng Zhang*, Hujun Bao*.
Survey and Evaluation of Monocular Visual-Inertial SLAM Algorithms for Augmented Reality. Virtual Reality & Intelligent Hardware, 1(4): 386-410, 2019.
[pdf]
[video]
[benchmark]
[evaluation tool]
Kangkan Wang, Guofeng Zhang, Shihong Xia. Templateless Non-Rigid Reconstruction and Motion Tracking With a Single RGB-D Camera. IEEE Transactions on Image Processing, 26(12): 5966 - 5979, 2017. [pdf][video]
Haomin Liu, Guofeng Zhang*, Hujun Bao. A Survey of Monocular Simultaneous Localization and Mapping. Journal of Computer-Aided Design & Computer Graphics, 28(6): 855 – 868, 2016 (in Chinese). (刘浩敏,章国锋,鲍虎军. 基于单目视觉的同时定位与地图构建方法综述. 计算机辅助设计与图形学学报, 28(6): 855 – 868, 2016.) [pdf]
Guofeng Zhang, Yi He, Weifeng Chen, Jiaya Jia and Hujun Bao*. Multi-Viewpoint Panorama Construction with Wide-Baseline Images. IEEE Transactions on Image Processing, 25(7):3099-3111, 2016.[pdf][supplementary document][video]
Hanqing Jiang , Guofeng Zhang*, Huiyan Wang and Hujun Bao. Spatio-Temporal Video Segmentation of Static Scenes and Its Applications. IEEE Transactions on Multimedia, 17(1):3-15, 2015.[pdf][supplementary document][video]
Guofeng Zhang, Hanqing Jiang, Jin Huang, Jiaya Jia, Tien-Tsin Wong, Kun Zhou, and Hujun Bao. Motion Imitation with a Handheld Camera. IEEE Transactions on Visualization and Computer Graphics (TVCG), 17(10): 1475-1486, 2011. [pdf][video]
Guofeng Zhang, Jiaya Jia, Wei Hua, and Hujun Bao. Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(3): 603-617, 2011.[pdf][video][more results]
Guofeng Zhang, Zilong Dong, Jiaya Jia, Liang Wan, Tien-Tsin Wong, and Hujun Bao. Refilming with Depth-Inferred Videos. IEEE Transactions on Visualization and Computer Graphics (TVCG), 15(5):828-840,2009.
[pdf][project]
Weicai Ye†, Xinyue Lan†, Shuo Chen, Yuhang Ming, Xinyuan Yu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang*.
PVO: Panoptic Visual Odometry.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
2023.
[pdf]
[source code]
[project]
[video]
Chong Bao†, Yinda Zhang†, Bangbang Yang†, Tianxing Fan, Zesong Yang, Hujun Bao, Guofeng Zhang*, Zhaopeng Cui*.
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
2023.
[pdf]
[source code]
[project]
[video]
Junjie Ni†, YiJin Li†, Zhaoyang Huang, Hongsheng Li, Hujun Bao, Zhaopeng Cui and Guofeng Zhang*.
PATS: Patch Area Transportation with Subdivision for Local Feature Matching.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
2023.
[pdf]
[source code]
[project]
[video]
Guanglin Li, Yifeng Li, Zhichao Ye, Qihang Zhang, Tao Kong, Zhaopeng Cui, Guofeng Zhang.
Generative Category-Level Shape and Pose Estimation with Semantic Primitives.
Conference on Robot Learning (CoRL),
2022.
[pdf]
[project]
[source code]
[supplementary]
[OpenReview]
Xingrui Yang†, Hai Li†, Hongjia Zhai, Yuhang Ming, Yuqian Liu, Guofeng Zhang*.
Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation.
International Symposium on Mixed and Augmented Reality (ISMAR),
2022.
[pdf]
[project]
[code]
Yijin Li, Xinyang Liu, Wenqi Dong, Han Zhou, Hujun Bao, Guofeng Zhang, Yinda Zhang*, Zhaopeng Cui*.
DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image.
Proceedings of the 17th European Conference on Computer Vision (ECCV),
2022.
[arxiv]
[supplementary]
[project]
[video]
Bangbang Yang†, Chong Bao†, Junyi Zeng, Hujun Bao, Yinda Zhang*, Zhaopeng Cui*, Guofeng Zhang*.
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing.
Proceedings of the 17th European Conference on Computer Vision (ECCV),
2022 (Oral).
[pdf]
[source code]
[project]
[video]
Boming Zhao†, Bangbang Yang†, Zhenyang Li, Zuoyue Li, Guofeng Zhang, Jiashu Zhao, Dawei Yin, Zhaopeng Cui*, Hujun Bao*.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation.
Proceedings of the ACM International Conference on Multimedia (MM),
2022 (Oral).
[pdf]
[source code]
[project]
[supplementary]
[video]
Jiaming Sun†, Zihao Wang†, Siyu Zhang†, Xingyi He, Hongcheng Zhao, Guofeng Zhang, Xiaowei Zhou.
OnePose: One-Shot Object Pose Estimation without CAD Models.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2022.
[pdf]
[source code]
[project]
[supplementary]
[video]
Luwei Yang†, Rakesh Shrestha†, Wenbo Li, Shuaicheng Liu, Guofeng Zhang, Zhaopeng Cui, Ping Tan.
SceneSqueezer: Learning to Compress Scene for Camera Relocalization.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2022.
(Oral presentation)
[pdf]
[supplementary]
Yan Xu, Kwan-Yee Lin, Guofeng Zhang, Xiaogang Wang, Hongsheng Li.
RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2022.
[pdf]
[source code]
[supplementary]
Haoyu Guo†, Sida Peng†, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou.
Neural 3D Scene Reconstruction with the Manhattan-world Assumption.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2022. (Oral presentation)
[pdf]
[source code]
[supplementary]
[project]
[video]
Danpeng Chen, Shuai Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Hujun Bao, Guofeng Zhang.
VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM.
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). 2022.
[pdf]
[video]
Zhichao Ye, Chong Bao, Xinyang Liu, Hujun Bao, Zhaopeng Cui, Guofeng Zhang.
Crossview Mapping with Graph-based Geolocalization on City-Scale Street Maps.
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). 2022.
[pdf]
[video]
Sandro Lombardi, Bangbang Yang, Tianxing Fan, Hujun Bao, Guofeng Zhang, Marc Pollefeys, Zhaopeng Cui*.
LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies.
Proceedings of the International Conference on 3D Vision (3DV),
2021 (Oral).
[pdf]
[project]
[supplementary]
[video]
Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, Guofeng Zhang*.
Graph-based Asynchronous Event Processing for Rapid Object Recognition.
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV),
2021.
[pdf]
[supplementary]
[video]
Jiaming Sun, Yiming Xie, Siyu Zhang, Linghao Chen, Guofeng Zhang, Hujun Bao, Xiaowei Zhou.
You Don't Only Look Once: Constructing Spatial-Temporal Memory for Integrated 3D Object Detection and Tracking.
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV),
2021.
[pdf]
[source code]
[project]
[supplementary]
[video]
Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, Zhaopeng Cui*.
Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering.
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV),
2021.
[pdf]
[source code]
[project]
[supplementary]
[talk]
[video]
Kangkan Wang, Huayu Zheng, Guofeng Zhang, Jian Yang.
Parametric Model Estimation for 3D Clothed Humans from Point Clouds.
International Symposium on Mixed and Augmented Reality (ISMAR), 2021.
[pdf]
Danpeng Chen, Nan Wang, Runsen Xu, Weijian Xie, Hujun Bao, Guofeng Zhang*.
RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes.
International Symposium on Mixed and Augmented Reality (ISMAR), 2021.
[pdf]
[source code]
[project]
[video]
Hai Li, Tianxing Fan, Hongjia Zhai, Zhaopeng Cui, Hujun Bao, Guofeng Zhang*.
BDLoc: Global Localization from 2.5D Building Map.
International Symposium on Mixed and Augmented Reality (ISMAR), 2021.
[pdf]
[video]
Zhaoyang Huang, Han Zhou, Yijin Li, Bangbang Yang, Yan Xu, Xiaowei Zhou, Hujun Bao, Guofeng
Zhang*, Hongsheng Li.
VS-Net: Voting with Segmentation for Visual Localization.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),
2021.
[pdf]
[source code]
[project]
[supplementary]
Weicai Ye, Hai Li, Tianxiang Zhang, Xiaowei Zhou, Hujun Bao, Guofeng Zhang*.
SuperPlane: 3D Plane Detection and Description from a Single Image.
IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2021.
[pdf]
[video]
[talk]
Haomin Liu, Mingxuan Jiang, Zhuang Zhang, Xiaopeng Huang, Linsheng Zhao, Meng Hang, Youji Feng, Hujun Bao, Guofeng Zhang*.
LSFB: A Low-cost and Scalable Framework for Building Large-Scale Localization Benchmark.
International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 219-224, 2020.
[pdf]
[video]
Hailin Yu, Weicai Ye, Youji Feng, Hujun Bao, Guofeng Zhang*.
Learning Bipartite Graph Matching for Robust Visual Localization.
International Symposium on Mixed and Augmented Reality (ISMAR), 2020.
[pdf]
[code]
Hai Li†, Weicai Ye†, Guofeng Zhang*, Sanyuan Zhang, Hujun Bao.
Saliency Guided Subdivision for Single-View Mesh Reconstruction.
International Conference on 3D Vision (3DV), 2020.
[pdf]
Kangkan Wang, Jin Xie, Guofeng Zhang, Lei Liu
, Jian Yang. Sequential 3D Human Pose and Shape Estimation from Point Clouds. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7275-7284, 2020. [pdf]
Zhichao Ye, Guofeng Zhang*, Hujun Bao. Efficient Covisibility-based Image Matching for Large-Scale SfM. IEEE International Conference on Robotics and Automation (ICRA), pp. 8616-8622, 2020. [pdf]
[code]
Zhaoyang Huang, Yan Xu, Jianping Shi, Xiaowei Zhou, Hujun Bao*, Guofeng Zhang*. Prior Guided Dropout for Robust Visual Localization in Dynamic Environments. IEEE International Conference on Computer Vision (ICCV), pp. 2791-2800, 2019. [pdf][source code]
Yan Xu, Xinge Zhu, Jianping Shi, Guofeng Zhang, Hujun Bao, Hongsheng Li. Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints. IEEE International Conference on Computer Vision (ICCV), pp. 2811-2820, 2019. [pdf]
Jinyu Li, Hujun Bao, and Guofeng Zhang*. Rapid and Robust Monocular Visual-Inertial Initialization with Gravity Estimation via Vertical Edges. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019. [pdf][source code]
Jinyu Li, Bangbang Yang, Kai Huang, Guofeng Zhang, and Hujun Bao*. Robust and Efficient Visual-Inertial Odometry with Multi-plane Priors. PRCV 2019, LNCS 11859, pp. 283–295, 2019. [pdf][video1video2][source code]
Haomin Liu, Mingyu Chen, Guofeng Zhang, Hujun Bao and Yingze Bao. ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM. IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2018.[pdf][source code]
Ming Hsiao, Eric Westman, Guofeng Zhang, Michael Kaess. Keyframe-based Dense Planar SLAM. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2017.[pdf]
Shuangli Zhang , Weijian Xie, Guofeng Zhang*, Hujun Bao, Michael Kaess. Robust Stereo Matching with Surface Normal Prediction. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2017.[pdf][video]
Haomin Liu, Guofeng Zhang*, Hujun Bao*. Robust Keyframe-based Monocular SLAM for Augmented Reality. International Symposium on Mixed and Augmented Reality (ISMAR), 2016.[pdf][video][software & datasets]
Wei Tan, Haomin Liu, Zilong Dong, Guofeng Zhang* and Hujun Bao. Robust Monocular SLAM in Dynamic Environments. International Symposium on Mixed and Augmented Reality (ISMAR), 2013.[pdf][video][talk slides][software]