Effective Eyebrow Matting with Domain Adaptation
Computer Graphics Forum (Special Issue of Pacific Graphics'2022), 2022, 41(7): 347-358.
Luyuan Wang, Hanyuan Zhang, Qinjie Xiao, Hao Xu, Chunhua Shen, and Xiaogang Jin
We create the first synthetic eyebrow matting dataset (a). This enables semi-supervised training of a domain adaptation eyebrow matting network. The network can learn domain-robust features from synthetic data (a) together with unlabeled real-world eyebrow images without using any real matting data and estimate high-quality eyebrow alpha matte (c) from a real RGB image (b) only without any prior. The eyebrow matting allows us to automatically remove the interference of eyebrows during the multi-view stereo (MVS) based 3D face reconstruction process, and therefore largely enhances the efficiency and efficacy of the reconstruction of eyebrow regions. Without eyebrow removal, the reconstructed eyebrow geometry (f) often induces noises and artifacts when fitting the eyebrow during 3D parametric face reconstruction (g), which requires very expensive manual repair in hours. In contrast, eyebrow matting facilitates the easy attainment of better geometry (h) and more faithful parameterization of the eyebrow region (i). Furthermore, our eyebrow matting method can be used for cosmetic design purposes such as eyebrow recoloring (d) and eyebrow replacement (e).
Abstract
We present the first
synthetic eyebrow matting datasets and a domain adaptation eyebrow matting
network for learning domain-robust feature representation using synthetic
eyebrow matting data and unlabeled in-the-wild images with adversarial
learning. Different from existing matting methods that may suffer from the
lack of ground-truth matting datasets, which are typically labor-intensive
to annotate or even worse, unable to obtain, we train the matting network in
a semi-supervised manner using synthetic matting datasets instead of
ground-truth matting data while achieving high-quality results.
Specifically, we first generate a large-scale synthetic eyebrow matting
dataset by rendering avatars and collect a real-world eyebrow image dataset
while maximizing the data diversity as much as possible. Then, we use the
synthetic eyebrow dataset to train a multi-task network, which consists of a
regression task to estimate the eyebrow alpha mattes and an adversarial task
to adapt the learned features from synthetic data to real data. As a result,
our method can successfully train an eyebrow matting network using synthetic
data without the need to label any real data. Our method can accurately
extract eyebrow alpha mattes from in-the-wild images without any additional
prior and achieves state-of-the-art eyebrow matting performance. Extensive
experiments demonstrate the superior performance of our method with both
qualitative and quantitative results.