Xiangjun Tang, Wenxin Sun, Yongliang Yang, Xiaogang Jin
Our parametric reshaping method allows users to reshape the portrait in a video footage easily by simply adjusting a reshaping parameter. Given an input portrait video sequence (second row), our approach can reshape the portrait in the video with weight-change such that the face appears thinner (the first row) or rounder (the third row), respectively.
Sharing short
personalized videos to various social media networks has become quite
popular in recent years. This raises the need for digital retouching of
portraits in videos. However, applying portrait image editing directly on
portrait video frames cannot generate smooth and stable video sequences. To
this end, we present a robust and easy-to-use parametric method to reshape
the portrait in a video to produce smooth retouched results. Given an input
portrait video, our method consists of two main stages: stabilized face
reconstruction, and continuous video reshaping. In the first stage, we start
by estimating face rigid pose transformations across video frames. Then we
jointly optimize multiple frames to reconstruct an accurate face identity,
followed by recovering face expressions over the entire video. In the second
stage, we first reshape the reconstructed 3D face using a parametric
reshaping model reflecting the weight change of the face, and then utilize
the reshaped 3D face to guide the warping of video frames. We develop a
novel signed distance function based dense mapping method for the warping
between face contours before and after reshaping, resulting in stable warped
video frames with minimum distortions. In addition, we use the 3D structure
of the face to correct the dense mapping to achieve temporal consistency. We
generate the final result by minimizing the background distortion through
optimizing a content-aware warping mesh. Extensive experiments show that our
method is able to create visually pleasing results by adjusting a simple
reshaping parameter, which facilitates portrait video editing for social
media and visual effects.