DEV Community

Cover image for PifuHD: 3D avatar from 1 image (well... almost)
Alexey Okhrimenko
Alexey Okhrimenko

Posted on

PifuHD: 3D avatar from 1 image (well... almost)

Recently, GDG DevParty Russia was held at which, in addition to talk on MagicString library, there was a report by Vasilika Klimova on the possibilities of WebGL in the browser β€œLet's dance?”. In it, she talked about how to create a 3D avatar, and then animate it directly in the browser. The problem is that the avatar is created using a huge device called "Shapify Booth". Cool if you have access and some extra money for this kind of entertainment, but what if not? :)

Thanks to the achievements of Machine Learning, we have an alternative. Facebook Research published a work called PifuHD. Using it, we can create a 3D avatar from single image 😱
To do this, find 1 full-body-length photograph of yourself.

Alt Text

Next, open Google Colab (via the link) (better to do this in Google Chrome) and follow the instructions. If you get confused, watch the video.

Literally in 5-10 minutes you will have your own 3D avatar from the picture.

Alt Text

But that is not all! Stitching 2 photos into one (front & back), thereby preparing the texture for our avatar

Alt Text

and watching a couple of videos about an easy way to texture a 3D model in Blender


We can create a textured model.

Alt Text

Having exported all this in FBX format and creating an archive in which the fbx file and texture file will lie, we can import this archive into Mixamo

Alt Text

In which we can choose the appropriate animation to poison it in Adobe Aero where we are already substituting our model in a suitable environment to get just such an animation πŸ˜…

Well, after a bit of training, I’ll finally deal with my inner self.

Of course, the quality of the models leaves much to be desired and sometimes even an obvious picture gives a huge number of artifacts. PifuHD is especially bad with hands.

Alt Text

And texturing/animating will take you more than 10 minutes, especially for the first time (it took me 6 hours), but with a little training you can make a lot of cool animations and maybe even a simple game :)

What a time to be alive! (c)

Subscribe to my twitter https://twitter.com/obenjiro and Telegram chat TensorflowJS to always be up to date with the latest ML https://t.me/tensorflowjs_en

Discussion (2)

Collapse
decajcd profile image
xsqll

How to use two pictures to texture a 3D model?

Collapse
obenjiro profile image
Alexey Okhrimenko Author

follow this tutorial but use merged picture (front and back) instead youtube.com/watch?v=kERzhJKYvGo&fe...