[Defense] Deep Learning based 3D Face Modeling with Fine Details from In-the-wild Images
Thursday, February 3, 2022
10:00 am - 12:00 pm
In
Partial
Fulfillment
of
the
Requirements
for
the
Degree
of
Doctor
of
Philosophy
Qixin
Deng
will
defend
his
proposal
Deep
Learning
based
3D
Face
Modeling
with
Fine
Details
from
In-the-wild
Images
Abstract
In this proposal, I first propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, I implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. Then I introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. The first work can augment an exist 3D face model with more fine-scale details, but to create a realistic human face model is not yet solved. To reconstruct an realistic face model from an in-the-wild image, I propose a convolutional neural network based framework to regress the face model from a single image in the wild. I designed novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting. The outputted face model includes dense 3D shape, head pose, expression, diffuse map, specular map, and the corresponding lighting conditions. Besides a carefully-designed ablation study, I also conduct direct comparison experiments to show that our method can outperform state-of-art methods both quantitatively and qualitatively.
Thursday,
February
3,
2022
10:00AM
-
12:00PM
CT
Online
via
MS
Teams
(click
link)
Dr. Zhigang Deng, dissertation advisor
Faculty, students and the general public are invited.
