[Defense] Deep Learning based 3D Face Modeling with Fine Details from In-the-wild Images
Thursday, February 3, 2022
10:00 am - 12:00 pm
will defend his proposal
Deep Learning based 3D Face Modeling with Fine Details from In-the-wild Images
In this proposal, I first propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, I implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. Then I introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. The first work can augment an exist 3D face model with more fine-scale details, but to create a realistic human face model is not yet solved. To reconstruct an realistic face model from an in-the-wild image, I propose a convolutional neural network based framework to regress the face model from a single image in the wild. I designed novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting. The outputted face model includes dense 3D shape, head pose, expression, diffuse map, specular map, and the corresponding lighting conditions. Besides a carefully-designed ablation study, I also conduct direct comparison experiments to show that our method can outperform state-of-art methods both quantitatively and qualitatively.
10:00AM - 12:00PM CT
Online via MS Teams (click link)
Dr. Zhigang Deng, dissertation advisor
Faculty, students and the general public are invited.