กก
Deep Shapely Portraits
กก
ACM Multimedia, October 12-16, 2020, Seattle, WA, United States.
Qinjie Xiao, Xiangjun Tang, You Wu, Leyang Jin, Yongliang Yang, and Xiaogang Jin
Given input portrait images with varying poses and expressions (bottom), our approach can automatically generate shapely portraits (top) that are better proportioned, by estimating the best reshaping parameter setting (called shapely degree) using deep learning.
Abstract
We present deep shapely portraits, a novel method
based on deep learning, to automatically reshape an input portrait to be
better proportioned and more shapely while keeping personal facial
characteristics. Different from existing methods that may suffer from
irrational face artifacts when dealing with portraits with large pose
variations or reshaping adjustments, we utilize dense 3D face
information and constraints instead of sparse facial landmarks based on
3D morphable models, resulting in better reshaped faces lying inrational
face space. To this end, we first estimate the best shapely degree for
the input portrait using a convolutional neural network (CNN) trained on
our newly developed ShapeFaceNet dataset. Then the best shapely degree
is used as the control parameter to reshape the 3D face reconstructed
from the input portrait image. After that, we render the reshaped 3D
face back to 2D and generate a seamless portrait image using a fast
image warping optimization. Our work can deal with pose and expression
free (PE-Free) portrait images and generate plausible shapely faces
without noticeable artifacts, which cannot be achieved by prior work. We
validate the effectiveness, efficiency, and robustness of the proposed
method by extensive experiments and user studies.