Effective Eyebrow Matting with Domain Adaptation

 

Computer Graphics Forum (Special Issue of Pacific Graphics'2022), 2022, 41(7): 347-358.

Luyuan Wang, Hanyuan Zhang, Qinjie Xiao, Hao Xu, Chunhua Shen, and  Xiaogang Jin

We create the first synthetic eyebrow matting dataset (a). This enables semi-supervised training of a domain adaptation eyebrow matting network. The network can learn domain-robust features from synthetic data (a) together with unlabeled real-world eyebrow images without using any real matting data and estimate high-quality eyebrow alpha matte (c) from a real RGB image (b) only without any prior. The eyebrow matting allows us to automatically remove the interference of eyebrows during the multi-view stereo (MVS) based 3D face reconstruction process, and therefore largely enhances the efficiency and efficacy of the reconstruction of eyebrow regions. Without eyebrow removal, the reconstructed eyebrow geometry (f) often induces noises and artifacts when fitting the eyebrow during 3D parametric face reconstruction (g), which requires very expensive manual repair in hours. In contrast, eyebrow matting facilitates the easy attainment of better geometry (h) and more faithful parameterization of the eyebrow region (i). Furthermore, our eyebrow matting method can be used for cosmetic design purposes such as eyebrow recoloring (d) and eyebrow replacement (e).

Abstract

We present the first synthetic eyebrow matting datasets and a domain adaptation eyebrow matting network for learning domain-robust feature representation using synthetic eyebrow matting data and unlabeled in-the-wild images with adversarial learning. Different from existing matting methods that may suffer from the lack of ground-truth matting datasets, which are typically labor-intensive to annotate or even worse, unable to obtain, we train the matting network in a semi-supervised manner using synthetic matting datasets instead of ground-truth matting data while achieving high-quality results. Specifically, we first generate a large-scale synthetic eyebrow matting dataset by rendering avatars and collect a real-world eyebrow image dataset while maximizing the data diversity as much as possible. Then, we use the synthetic eyebrow dataset to train a multi-task network, which consists of a regression task to estimate the eyebrow alpha mattes and an adversarial task to adapt the learned features from synthetic data to real data. As a result, our method can successfully train an eyebrow matting network using synthetic data without the need to label any real data. Our method can accurately extract eyebrow alpha mattes from in-the-wild images without any additional prior and achieves state-of-the-art eyebrow matting performance. Extensive experiments demonstrate the superior performance of our method with both qualitative and quantitative results.

Download

PDF, 27.1 MB Video, 126.0 MB