DEV Community

loading...

How to display 3D humanoid avatar with React

Tadao Iseki
Frontend dev
・4 min read

Introduction

3DCG and VR technology are used in various places and become familiar with us. And the same phenomenon occurs on web browser. Today I introduce VRM and how to display it with React and @pixiv/three-vrm.

What is VRM?

VRM is a file format for handling 3D humanoid avatar (3D model) data for VR applications. If you have an avatar that conforms to VRM, you can enjoy various applications that require a 3D avatar.

What is @pixiv/three-vrm?

GitHub logo pixiv / three-vrm

Use VRM on Three.js

@pixiv/three-vrm is a JavaScript library to use VRM on Three.js. This enables to render VRM on web applications like VRoid Hub.

VRoid Hub

Prepare VRM

At first, you need to download VRM from VRoid Hub.

  1. Search VRM models by tags.
  2. Select your favorite model.
  3. Move to model page and download by clicking "Use this model"

Set up project

$ npx create-react-app three-vrm-sample
$ cd three-vrm-sample/
$ yarn add @pixiv/three-vrm three react-three-fiber
<!DOCTYPE html>
<html>
  <head>
    <title>@pixiv/three-vrm sample</title>
    <style>
      html,
      body {
        background-color: #000;
        color: #fff;
        margin: 0;
        width: 100vw;
        height: 100vh;
      }

      #root {
        width: 100%;
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

import React from 'react'
import ReactDOM from 'react-dom'

const App = () => null

ReactDOM.render(<App />, document.getElementById('root'))

Add VRM Loader

We can load VRM with GLTFLoader because VRM is similar format to GLTF.

import { VRM } from '@pixiv/three-vrm'
import { useRef, useState } from 'react'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      setVrm(vrm)
    })
  }

  return { vrm, loadVrm }
}

Display VRM with react-three-fiber

react-three-fiber is a React renderer for Three.js. You can use Three.js declaratively with it. I use the following three elements this time.

  • <Canvas>: Wrapper element for react-three-fiber elements
  • <spotLight>: Light element to illuminate objects
  • <primitive>: 3D object element

When you input a VRM file, handleFileChange() create object url and load VRM.

import React from 'react'
import { Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

Result:

Result

Improve appearance

VRM model in the result is small and facing the other side. You want to see it closer and its face, don't you?

1. Create a new camera from THREE.PerspectiveCamera and set its position.

Note:
useThree gives you access to all the basic THREE objects like gl, scene, camera, clock and so on...

import React, { useEffect, useRef } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

2. Rotate and look at camera

Assign camera to vrm.lookAt.target and rotate vrm 180°.

import { VRM } from '@pixiv/three-vrm'
import { useEffect, useRef, useState } from 'react'
import { useThree } from 'react-three-fiber'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

Final Code:

import { VRM } from '@pixiv/three-vrm'
import ReactDOM from 'react-dom'
import React, { useEffect, useRef, useState } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

ReactDOM.render(<App />, document.getElementById('root'))

Result:

Result

Looks good 👍

Conclusion

VRM will be used more widely in the future. I hope this article will help you when you need to use VRM with React.

@pixiv/three-vrm has more features, so if you are interested, please read the documentation and try it out.

If you have any problems or questions, please write comment or reply to my Twitter account.

Sample Repository:

GitHub logo saitoeku3 / three-vrm-sample

👤Sample repository using @pixiv/three-vrm with React

Discussion (2)

Collapse
drcmda profile image
Paul Henschel

nice tutorial, never heard of vrm before, this will be useful!

saw a tiny glitch, some objs are initialized in a useRef, they get re-created every render. better do: const [myObj] = useState(() => new THREE.Something()), this is guaranteed to be created only once.

Collapse
saitoeku3 profile image
Tadao Iseki Author

Thanks for your excellent comment! I just try it now :D

Forem Open with the Forem app