photo

Guofeng Zhang

Full Professor , State Key lab of CAD&CG, College of Computer Science, Zhejiang University

Tel: +86-571-88208701    Fax: +86-571-88206680

Email: zhangguofeng@zju.edu.cn, zhangguofeng@cad.zju.edu.cn
Office: East Building 1A-509, State Key Lab of CAD&CG, ZijinGang Campus, Zhejiang University

Home Page: http://www.cad.zju.edu.cn/home/gfzhang


Dr. Zhang now is a full professor at Zhejiang University. He received the B.S. and Ph.D. degree in computer science and technology from Zhejiang University in 2003 and 2009, respectively. He is a core member of Computer Vision Group at State Key Lab of CAD&CG, Zhejiang University.

Research Interests:

  • SfM/SLAM & 3D Reconstruction

  • Video Enhancement & Editing
  • Augmented Reality

News:

July 2018: We have released the source code of ENFT-SfM, SegmentBA, EIBA and ICE-BA. Please visit https://github.com/zju3dv/ for more details.

Sep. 2016: RKSLAM released : RKSLAM 1.0 has been released. Please visit http://www.zjucvg.net/rkslam/rkslam.html for more details.

 

RKSLAM: Robust Keyframe-based Monocular SLAM for AR

 

RKSLAM

RKSLAM is a real-time monocular simultaneous localization and mapping system which can robustly work in challenging cases, such as fast motion and strong rotation. It can run real-time on a mobile device and outperform state-of-the-art systems (e.g. ORB-SLAM, PTAM, LSD-SLAM) in challenging cases of fast motion and strong rotation.

LS-ACTS: Large-Scale Automatic Camera Tracking System

 

LS-ACTS

LS-ACTS is a robust and efficient structure-from-motion system which can recover camera motion and 3D scene structure from large videos/sequences datasets. Compared to our previous SfM system ACTS, it is much faster (near real-time in a normal desktop PC) and can handle multiple extremely long sequences (over 100K frames).

RDSLAM: Robust Dynamic Simultaneous Localization and Mapping

 

RDSLAM system

RDSLAM is a real-time simultaneous localization and mapping system which allows parts of the scene to be dynamic or the whole scene to gradually change. Compared to PTAM, RDSLAM not only can robustly work in dynamic environments, but also can handle a larger scale scene (the number of the reconstructed 3D points can be tens of thousands). It is the basis for many applications, such as real-time 3D reconstruction and augmented reality.

ACTS: Automatic Camera Tracking System

 

ACTS is an automatic camera tracking system which can recover camera motion and 3D scene structure from videos and film sequences, providing the ease of automatic tracking. It can track all kinds of the camera motion efficiently and stably, which can be rotational or free-moving. Especially, the long sequences with varying focal length can be handled in a robust way. Besides camera motion, it also can recover accurate and dense depth maps now.

We will actively update the program and add more and more advanced or extra functions along with our published papers in the future.


Publications:

  Technical Reports:

RKD-SLAM
Haomin Liu, Chen Li, Guojun Chen, Guofeng Zhang*, Michael Kaess and Hujun Bao. Robust Keyframe-based Dense SLAM with an RGB-D Camera. arXiv preprint arXiv:1711.05166, 2017. [arXiv report][source code]
ENFT Guofeng Zhang, Haomin Liu, Zilong Dong, Jiaya Jia, Tien-Tsin Wong and Hujun Bao. ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion. Technical Report (arXiv:1510.08012), October, 2015. [arXiv report][video][software]
motion imitation Hanqing Jiang, Guofeng Zhang, and Hujun Bao. Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes. Technical Report, Zhejiang University, September, 2010. [pdf][video 1, 2, 3]

 

  Selected Journal Publications:

mobile3dscanner
Xiaojun Xiang, Hanqing Jiang, Guofeng Zhang, Yihao Yu, Chenchen Li, Xingbin Yang, Danpeng Chen, Hujun Bao*. Mobile3DScanner: An Online 3D Scanner for High-quality Object Reconstruction with a Mobile Device. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021 (ISMAR Best Journal Paper Nominee). [pdf] [project page] [video-1] [video-2]
DispRCNN Linghao Chen, Jiaming Sun, Yiming Xie, Siyu Zhang, Qing Shuai, Qinhong Jiang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou. Shape Prior Guided Instance Disparity Estimation for 3D Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. [pdf][video][source code]
Learning Dense...
Kangkan Wang*, Guofeng Zhang, Huayu Zheng, Jian Yang. Learning Dense Correspondences for Non-Rigid Point Clouds With Two-Stage Regression. IEEE Transactions on Image Processing (TIP), 30: 8468-8482, 2021. [pdf]
NIID-Net
Jundan Luo, Zhaoyang Huang, Yijin Li, Xiaowei Zhou, Guofeng Zhang, Hujun Bao*. NIID-Net: Adapting Surface Normal Knowledge for Intrinsic Image Decomposition in Indoor Scenes. IEEE Transactions on Visualization and Computer Graphics (TVCG), 26(12): 3434-3445, 2020. [pdf] [supplementary document] [video] [source code]
Mobile-3D-Recon
Xingbin Yang, Liyang Zhou, Hanqing Jiang, Zhongliang Tang, Yuanbo Wang, Hujun Bao, Guofeng Zhang*. Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone. IEEE Transactions on Visualization and Computer Graphics (TVCG), 26(12): 3446-3556, 2020 (ISMAR Best Paper Award). [pdf] [project page] [video-1] [video-2]
vislam_bench
Jinyu Li, Bangbang Yang, Danpeng Chen, Nan Wang, Guofeng Zhang*, Hujun Bao*. Survey and Evaluation of Monocular Visual-Inertial SLAM Algorithms for Augmented Reality. Virtual Reality & Intelligent Hardware, 1(4): 386-410, 2019. [pdf] [video] [benchmark] [evaluation tool]
Nonrigid 3DReconstruction
Kangkan Wang, Guofeng Zhang, Shihong Xia. Templateless Non-Rigid Reconstruction and Motion Tracking With a Single RGB-D Camera. IEEE Transactions on Image Processing, 26(12): 5966 - 5979, 2017. [pdf][video]
  Haomin Liu, Guofeng Zhang*, Hujun Bao. A Survey of Monocular Simultaneous Localization and Mapping. Journal of Computer-Aided Design & Computer Graphics, 28(6): 855 – 868, 2016 (in Chinese). (刘浩敏,章国锋,鲍虎军. 基于单目视觉的同时定位与地图构建方法综述. 计算机辅助设计与图形学学报, 28(6): 855 – 868, 2016.) [pdf]
ENFT
Guofeng Zhang*, Haomin Liu, Zilong Dong, Jiaya Jia, Tien-Tsin Wong, and Hujun Bao*. Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion. IEEE Transactions on Image Processing, 25(12): 5957 - 5970, 2016.[pdf][arXiv report][video][software][source code]
Pano Stitching Guofeng Zhang, Yi He, Weifeng Chen, Jiaya Jia and Hujun Bao*. Multi-Viewpoint Panorama Construction with Wide-Baseline Images. IEEE Transactions on Image Processing, 25(7):3099-3111, 2016.[pdf][supplementary document][video]
TMM paper Hanqing Jiang , Guofeng Zhang*, Huiyan Wang and Hujun Bao. Spatio-Temporal Video Segmentation of Static Scenes and Its Applications. IEEE Transactions on Multimedia, 17(1):3-15, 2015.[pdf][supplementary document][video]
3D Reconstruction with Kinect Kangkan Wang, Guofeng Zhang* and Hujun Bao. Robust 3D Reconstruction with an RGB-D Camera. IEEE Transactions on Image Processing, 23(11):4893-4906, 2014.[pdf][video]
realtime camera tracking Zilong Dong, Guofeng Zhang*, Jiaya Jia, and Hujun Bao. Efficient Keyframe-Based Real-Time Camera Tracking. Computer Vision and Image Understanding, 118: 97-110, 2014.[pdf][video]
motion imitation Guofeng Zhang, Hanqing Jiang, Jin Huang, Jiaya Jia, Tien-Tsin Wong, Kun Zhou, and Hujun Bao. Motion Imitation with a Handheld Camera. IEEE Transactions on Visualization and Computer Graphics (TVCG), 17(10): 1475-1486, 2011. [pdf][video]
motion cut Guofeng Zhang, Jiaya Jia, Wei Hua, and Hujun Bao. Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(3): 603-617, 2011.[pdf][video][more results]
refilming Guofeng Zhang, Zilong Dong, Jiaya Jia, Liang Wan, Tien-Tsin Wong, and Hujun Bao. Refilming with Depth-Inferred Videos. IEEE Transactions on Visualization and Computer Graphics (TVCG), 15(5):828-840,2009. [pdf][project]
Consistent Depth

Guofeng Zhang, Jiaya Jia, Tien-Tsin Wong, and Hujun Bao. Consistent Depth Maps Recovery from a Video Sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 31(6):974-988, 2009. [pdf][application][3D video data][software]

Stereo Video

Guofeng Zhang, Wei Hua, Xueying Qin, Tien-Tsin Wong, and Hujun Bao. Stereoscopic Video Synthesis from a Monocular Video. IEEE Transactions on Visualization and Computer Graphics (TVCG), 13(4): 686-696 (2007).[pdf][video]

 


    Selected Conference Publications:

Sandro Lombardi, Bangbang Yang, Tianxing Fan, Hujun Bao, Guofeng Zhang, Marc Pollefeys, Zhaopeng Cui*. LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies. Proceedings of the International Conference on 3D Vision (3DV), 2021 (oral). [pdf] [source code] [project] [supplementary] [video]
ICCV 2021
Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, Guofeng Zhang*. Graph-based Asynchronous Event Processing for Rapid Object Recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [pdf] [supplementary] [video]
Jiaming Sun, Yiming Xie, Siyu Zhang, Linghao Chen, Guofeng Zhang, Hujun Bao, Xiaowei Zhou. You Don't Only Look Once: Constructing Spatial-Temporal Memory for Integrated 3D Object Detection and Tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [pdf] [source code] [project] [supplementary] [video]
Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, Zhaopeng Cui*. Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [pdf] [source code] [project] [supplementary] [talk] [video]
ISMAR2021_3DClothedHumans
Kangkan Wang, Huayu Zheng, Guofeng Zhang, Jian Yang. Parametric Model Estimation for 3D Clothed Humans from Point Clouds. International Symposium on Mixed and Augmented Reality (ISMAR), 2021. [pdf]
ISMAR2021_RNIN-VIO
Danpeng Chen, Nan Wang, Runsen Xu, Weijian Xie, Hujun Bao, Guofeng Zhang*. RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes. International Symposium on Mixed and Augmented Reality (ISMAR), 2021. [pdf] [project] [video]
ISMAR2021_BDLoc
Hai Li, Tianxing Fan, Hongjia Zhai, Zhaopeng Cui, Hujun Bao, Guofeng Zhang*. BDLoc: Global Localization from 2.5D Building Map. International Symposium on Mixed and Augmented Reality (ISMAR), 2021. [pdf] [video]
CVPR 2021
Zhaoyang Huang, Han Zhou, Yijin Li, Bangbang Yang, Yan Xu, Xiaowei Zhou, Hujun Bao, Guofeng Zhang*, Hongsheng Li. VS-Net: Voting with Segmentation for Visual Localization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), 2021. [pdf] [source code] [project] [supplementary]
IROS 2021
Xiangyu Liu, Weicai Ye, Chaoran Tian, Zhaopeng Cui, Hujun Bao, Guofeng Zhang*. Coxgraph: Multi-Robot Collaborative, Globally Consistent, Online Dense Reconstruction System. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021 (Best Paper Finalist Award on Safety, Security, and Rescue Robotics in memory of Motohiro Kisoi). [pdf] [source code] [video] [presentation]
VR 2021
Weicai Ye, Hai Li, Tianxiang Zhang, Xiaowei Zhou, Hujun Bao, Guofeng Zhang*. SuperPlane: 3D Plane Detection and Description from a Single Image. IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2021. [pdf] [video] [talk]
LSBF
Haomin Liu, Mingxuan Jiang, Zhuang Zhang, Xiaopeng Huang, Linsheng Zhao, Meng Hang, Youji Feng, Hujun Bao, Guofeng Zhang*. LSFB: A Low-cost and Scalable Framework for Building Large-Scale Localization Benchmark. International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 219-224, 2020. [pdf] [video]
BGNet
Hailin Yu, Weicai Ye, Youji Feng, Hujun Bao, Guofeng Zhang*. Learning Bipartite Graph Matching for Robust Visual Localization. International Symposium on Mixed and Augmented Reality (ISMAR), 2020. [pdf]
3dv 2020
Hai Li, Weicai Ye, Guofeng Zhang*, Sanyuan Zhang, Hujun Bao. Saliency Guided Subdivision for Single-View Mesh Reconstruction. International Conference on 3D Vision (3DV), 2020. [pdf]
CVPR 2020-Human Pose and Shape Estimation
Kangkan Wang, Jin Xie, Guofeng Zhang, Lei Liu , Jian Yang. Sequential 3D Human Pose and Shape Estimation from Point Clouds. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7275-7284, 2020. [pdf]
ICRA 2020-Image Matching
Zhichao Ye, Guofeng Zhang*, Hujun Bao. Efficient Covisibility-based Image Matching for Large-Scale SfM. IEEE International Conference on Robotics and Automation (ICRA), pp. 8616-8622, 2020. [pdf]
ICCV 2019-Robust Visual Localization
Zhaoyang Huang, Yan Xu, Jianping Shi, Xiaowei Zhou, Hujun Bao*, Guofeng Zhang*. Prior Guided Dropout for Robust Visual Localization in Dynamic Environments. IEEE International Conference on Computer Vision (ICCV), pp. 2791-2800, 2019. [pdf][source code]
ICCV 2019-Depth Completion
Yan Xu, Xinge Zhu, Jianping Shi, Guofeng Zhang, Hujun Bao, Hongsheng Li. Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints. IEEE International Conference on Computer Vision (ICCV), pp. 2811-2820, 2019. [pdf]
IROS 2019
Jinyu Li, Hujun Bao, and Guofeng Zhang*. Rapid and Robust Monocular Visual-Inertial Initialization with Gravity Estimation via Vertical Edges. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019. [pdf][source code]
PRCV 2019
Jinyu Li, Bangbang Yang, Kai Huang, Guofeng Zhang, and Hujun Bao*. Robust and Efficient Visual-Inertial Odometry with Multi-plane Priors. PRCV 2019, LNCS 11859, pp. 283–295, 2019. [pdf][video1 video2][source code]
ICE-BA Haomin Liu, Mingyu Chen, Guofeng Zhang, Hujun Bao and Yingze Bao. ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.[pdf][source code]
KDP-SLAM Ming Hsiao, Eric Westman, Guofeng Zhang, Michael Kaess. Keyframe-based Dense Planar SLAM. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2017.[pdf]
RSM-SNP Shuangli Zhang , Weijian Xie, Guofeng Zhang*, Hujun Bao, Michael Kaess. Robust Stereo Matching with Surface Normal Prediction. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2017.[pdf][video]
RKSLAM Haomin Liu, Guofeng Zhang*, Hujun Bao*. Robust Keyframe-based Monocular SLAM for Augmented Reality. International Symposium on Mixed and Augmented Reality (ISMAR), 2016.[pdf][video][software & datasets]
3DV 2014 Weifeng Chen, Guofeng Zhang*, Xiaojun Xiang, Jiaya Jia and Hujun Bao. High-Quality Depth Recovery via Interactive Multi-View Stereo. 3DV, 2014.[pdf][supplementary document][video]
Dynamic SLAM Wei Tan, Haomin Liu, Zilong Dong, Guofeng Zhang* and Hujun Bao. Robust Monocular SLAM in Dynamic Environments. International Symposium on Mixed and Augmented Reality (ISMAR), 2013.[pdf][video][talk slides][software]
eccv 2012 Hanqing Jiang, Haomin Liu, Ping Tan, Guofeng Zhang*, and Hujun Bao. 3D Reconstruction of Dynamic Scenes with Multiple Handheld Cameras. European Conference on Computer Vision (ECCV), 2012.[pdf][supplementary document][video]
TriDepth Wenzhuo Yang, Guofeng Zhang*, Hujun Bao, Jiwon Kim and Ho Young Lee. Consistent Depth Maps Recovery from a Trinocular Video Sequence. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.[pdf][supplementary document][video]
multi-body MVS Guofeng Zhang, Jiaya Jia, and Hujun Bao. Simultaneous Multi-Body Stereo and Segmentation. IEEE International Conference on Computer Vision (ICCV), 2011.[pdf][video]
Guofeng Zhang, Zilong Dong, Jiaya Jia, Tien-Tsin Wong, and Hujun Bao. Efficient Non-Consecutive Feature Tracking for Structure-from-Motion. European Conference on Computer Vision (ECCV), 2010.[pdf][video]
Zilong Dong, Guofeng Zhang*, Jiaya Jia, and Hujun Bao. Keyframe-Based Real-Time Camera Tracking. IEEE International Conference on Computer Vision (ICCV), pages 1538-1545, 2009. [pdf][video][project]

Video Depth

Guofeng Zhang, Jiaya Jia, Tien-Tsin Wong, and Hujun Bao.  Recovering Consistent Video Depth Maps via Bundle Optimization. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008(oral). [pdf] [application][3D video data]

Motion Cut

Guofeng Zhang, Jiaya Jia, Wei Xiong, Tien-Tsin Wong, Pheng-Ann Heng, and Hujun Bao. Moving Object Extraction with a Hand-held Camera.  IEEE International Conference on Computer Vision (ICCV), 2007. [pdf]

Camera Tracking

Guofeng Zhang, Xueying Qin, Wei Hua, Tien-Tsin Wong, Pheng-Ann Heng, and Hujun Bao. Robust Metric Reconstruction from Challenging Video Sequences. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007. [pdf][video][software]

 

visitors since Feb. 6th, 2020