Apple Researchers Come Up With FaceLit To Create 3D Faces From 2D Pictures

Apple has been awfully quiet while an AI revolution has taken over rest of big tech, but it’s quietly coming up with AI research all the same.

Researchers from Apple have come up with a framework that generates a 3D face learned purely from 2D images. :We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation,” the paper says.

“Unlike existing works that require careful capture setup or human labor, we rely on off-the-shelf pose and illumination estimators. With these estimates, we incorporate the Phong reflectance model in the neural volume rendering framework. Our model learns to generate shape and material properties of a face such that, when rendered according to the natural statistics of pose and illumination, produces photorealistic face images with multiview 3D and illumination consistency. Our method enables photorealistic generation of faces with explicit illumination and view controls on multiple datasets – FFHQ, MetFaces and CelebA-HQ. We show state-of-the-art photorealism among 3D aware GANs on FFHQ dataset achieving an FID score of 3.5,” the paper says.

The demo shows Apple creating 3D faces, which are lit up in different ways depending on how they’re lit. The research could come in handy in a variety of ways, either for producing animated emojis for the iPhone, or even for Apple’s

But the paper shows that Apple is still up and kicking in the AI race. Google, or course, has several AI initiatives, and Meta has also released some impressive open-source projects. Many experts, though, think Apple might be the dark horse — it’s already creating chips to enable models to run on personal computers without the internet, and experts, such as Stability AI’s Emad Mostaque seem to believe that it will be the eventual winner in the AI race. And with the company now releasing some of its research, it might be signaling that it’s time for it to stand up and be counted.