PifuHD: 3D avatar from 1 image (well... almost)

#3d #ar #machinelearning #webdev

Recently, GDG DevParty Russia was held at which, in addition to talk on MagicString library, there was a report by Vasilika Klimova on the possibilities of WebGL in the browser “Let's dance?”. In it, she talked about how to create a 3D avatar, and then animate it directly in the browser. The problem is that the avatar is created using a huge device called "Shapify Booth". Cool if you have access and some extra money for this kind of entertainment, but what if not? :)

Thanks to the achievements of Machine Learning, we have an alternative. Facebook Research published a work called PifuHD. Using it, we can create a 3D avatar from single image 😱
To do this, find 1 full-body-length photograph of yourself.

Next, open Google Colab (via the link) (better to do this in Google Chrome) and follow the instructions. If you get confused, watch the video.

Literally in 5-10 minutes you will have your own 3D avatar from the picture.

But that is not all! Stitching 2 photos into one (front & back), thereby preparing the texture for our avatar

and watching a couple of videos about an easy way to texture a 3D model in Blender

We can create a textured model.

Having exported all this in FBX format and creating an archive in which the fbx file and texture file will lie, we can import this archive into Mixamo

In which we can choose the appropriate animation to poison it in Adobe Aero where we are already substituting our model in a suitable environment to get just such an animation 😅

Well, after a bit of training, I’ll finally deal with my inner self.

Of course, the quality of the models leaves much to be desired and sometimes even an obvious picture gives a huge number of artifacts. PifuHD is especially bad with hands.

And texturing/animating will take you more than 10 minutes, especially for the first time (it took me 6 hours), but with a little training you can make a lot of cool animations and maybe even a simple game :)

What a time to be alive! (c)

Subscribe to my twitter https://twitter.com/obenjiro and Telegram chat TensorflowJS to always be up to date with the latest ML https://t.me/tensorflowjs_en