Logo image
Back
Towards high-fidelity facial UV map generation in real-world
Journal article   Peer reviewed

Towards high-fidelity facial UV map generation in real-world

Yuanming Li, Jeong-gi Kwak, Bon-hwa Ku, David Han and Hanseok Ko
Pattern recognition letters, v 180, pp 68-74
Apr 2024
DOI: https://doi.org/10.1016/j.patrec.2024.02.023

Abstract

3D face reconstruction 3DMM Multi-view face image UV map
We present a framework for completing high-fidelity 3D facial UV maps from single-face image. Despite the success of Generative Adversarial Networks (GANs) in this area, generating accurate UV maps from in-the-wild images remains challenging. Our approach involves a novel network called “Map and Edit” that combines a 2D generative model and a 3D prior to explicitly control the generation of multi-view faces. We use an indirect method to address domain gap issues between rendered and real images, which improves the identity consistency of the generated multi-view facial images. We also leverage synthesized multi-view images and predicted 3D information to produce texture-rich and high-resolution facial UV maps. Our model is self-supervised and does not require manual annotations or datasets. Experimental results demonstrate the effectiveness of our framework in reconstructing high-fidelity UV maps with accurate, fine details. Overall, our approach provides a promising solution to the challenges of 3D facial UV map completion from in-the-wild images. •A novel scheme for generating high-fidelity face UV maps from single portrait.•Map and Edit network allows precise facial pose control with StyleGAN methods.•We address the domain gap in 3DMM StyleGAN edits due to photometric loss.•Map and Edit’ boosts UV map fidelity and speed for out-of-domain images.

Metrics

1 Record Views

Details

Logo image