Omher

Posted on Apr 18, 2022

From Desktop 3d Apps to Web 3d Apps using Blender and React

#react #threejs #javascript #webdev

In this tutorial, I will walk you thru the steps to create a 3d react application with some interactivity so in the final you will have something like this

What is Blender? - Simply Explained
Create React App
Install dependencies
Export blender asset
Compress asset
Convert asset to JSX component
Integrate new component
Enhanced component and functionality
- Adding some style
- Install dependency
- Edit React Components
Resources
Appendix

Before you start

You will need to have the following installed or configured and know at least the basics of using them before proceeding.

NodeJS installed (preferable > 12)
Basic Knowledge in React
Previous use of create-react-app
Not mandatory, but some basic knowledge of using blender 3d app to understand the concept of mesh and material

What is Blender? Simply Explained

This tutorial is not a blender tutorial, so that it will be a short explanation.
Blender is a free, open-source 3D creation suite. With a strong foundation of modeling capabilities, there's also robustly texturing, rigging, animation, lighting, and other tools for complete 3D creation.

Source: Spring - Blender Open Movie Blender, Animation Studio via YouTube

Create React App

npx create-react-app cra-fiber-threejs
npm run start

If everything works successfully, you can navigate to: http://localhost:3000/, and you will see a React App

Install dependencies

Install gltf-pipeline; this will help you to optimize our glTF, meaning smaller for the web; this is installed globally

npm install -g gltf-pipeline

Install @react-three dependencies for our project, navigate to cra-fiber-threejs folder and run

npm i @react-three/drei
npm i @react-three/fiber

Export blender asset

Open blender program with you're created, 3d model
if you have installed blender and created a 3d modeling, in case you didn't, take a look in the optional step

Optional

If you have installed blender but didn't create any model, here you have the one I'm using in the tutorial
If you didn't install blender and want the compressed glb file here, you can download it.

Compress asset

The file we exported from the previous step, some times are significant, and they are not optimized for the web, so we need to compress it
Navigate where you saved the .glb file (from the previous step) and run the following command:

gltf-pipeline -i <input file glb> -o <output glb> --draco.compressionLevel=10
e.g:
gltf-pipeline -i shoe.glb -o ShoeModelDraco.glb --draco.compressionLevel=10

Convert asset to `JSX` component

To start interacting with our 3d model, we need to convert it to a JSX component using gltfjsx. You can read more here. gltfjsx - Turns GLTFs into JSX components)

Navigate where you saved the .glb file from the previous step and run the following command:

npx gltfjsx <outputed glb from previus step>
e.g. npx gltfjsx ShoeModelDraco.glb

The output will be a js file with content similar to:

/*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'

export default function Model({ ...props }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb')
  return (
    <group ref={group} {...props} dispose={null}>
      <mesh geometry={nodes.shoe.geometry} material={materials.laces} />
      <mesh geometry={nodes.shoe_1.geometry} material={materials.mesh} />
      <mesh geometry={nodes.shoe_2.geometry} material={materials.caps} />
      <mesh geometry={nodes.shoe_3.geometry} material={materials.inner} />
      <mesh geometry={nodes.shoe_4.geometry} material={materials.sole} />
      <mesh geometry={nodes.shoe_5.geometry} material={materials.stripes} />
      <mesh geometry={nodes.shoe_6.geometry} material={materials.band} />
      <mesh geometry={nodes.shoe_7.geometry} material={materials.patch} />
    </group>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')

The output It's a React component with all the meshed/materials mapped ready to work
If you worked with blender, you can see that it has mapped all its meshes objects and all its materials
This component can now be dropped into your scene. It is asynchronous and, therefore, must be wrapped into <Suspense> which gives you complete control over intermediary loading-fallbacks and error handling.

Integrate new component

Go the project you created using create-react-app
Copy your new file created in step "Convert asset to JSX component" e.g. ShoeModelDraco.js to src/ folder
Create a new file for your new component and called it BlenderScene.js, this file will include for the simplicity also some logic and the Scene components, in a real application you will want to separate them in different files/components, copy the following code:

import React, { Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import Model from './ShoeModelDraco'
function Scene() {
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>

  );
}

export default BlenderScene;

Copy into the public folder the .glb output file from step "Export blender asset," in my case: ShoeModelDraco.glb
Use the BlenderScene component you just created, open the App.js file, and import it something like:

import './App.css';
import BlenderScene from './BlenderScene';

function App() {
  return (
    <BlenderScene /> 
  );
}

export default App;

If everything runs successfully, you should see your 3d model in the browser, something like this:

The only interactivity that you have it's that you can spin the 3d model, and that's it,
In the following steps, we will:
- Add more fun/complex interactivity
- Display nicer in the browser
- In the resources part, you can find a link for the branch with the code until this step

Enhanced component and functionality

If you are reading here, kudos 💪🏼.
You are almost done 🥵; you have your 3d model in the browser 🎉, but you saw, it's not very interesting and boring; let's start adding cool stuff 😎.

Disclaimer: The following code it's not production-ready, and I did some hacks and also not best practices when writing the components

Adding some style

Open the App.css file and add in the end of it the following:

#root {
  position: relative;
  margin: 0;
  padding: 0;
  overflow: hidden;
  outline: none;
  width: 100vw;
  height: 100vh;
}
.scene {
    height: 500px;
    padding: 100px;

}

Install dependency

We will install react-colorful, a tiny color picker component for React and Preact apps. We will use it for choosing colors

npm i react-colorful

Edit React Components

Open ShoeModelDraco.js file and copy the following code
We add functionality to work with the mouse when the user clicks on our model
We add state to know which part of the model was selected

    /*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'
import { useFrame } from '@react-three/fiber'
export default function Model({ props, currentState, setCurrentState, setHover }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb');
  // Animate model
  useFrame(() => {
    const t = performance.now() / 1000
    group.current.rotation.z = -0.2 - (1 + Math.sin(t / 1.5)) / 20
    group.current.rotation.x = Math.cos(t / 4) / 8
    group.current.rotation.y = Math.sin(t / 4) / 8
    group.current.position.y = (1 + Math.sin(t / 1.5)) / 10
  })
  return (
    <>
      <group
      ref={group} {...props}
      dispose={null}
      onPointerOver={(e) => {
        e.stopPropagation();
        setHover(e.object.material.name);
      }}
      onPointerOut={(e) => {
        e.intersections.length === 0 && setHover(null);
      }}
      onPointerMissed={() => {
        setCurrentState(null);
      }}
      onClick={(e) => {
        e.stopPropagation();
        setCurrentState(e.object.material.name);
      }}>
      <mesh receiveShadow castShadow geometry={nodes.shoe.geometry} material={materials.laces} material-color={currentState.items.laces} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_1.geometry} material={materials.mesh} material-color={currentState.items.mesh} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_2.geometry} material={materials.caps} material-color={currentState.items.caps} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_3.geometry} material={materials.inner} material-color={currentState.items.inner} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_4.geometry} material={materials.sole} material-color={currentState.items.sole} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_5.geometry} material={materials.stripes} material-color={currentState.items.stripes} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_6.geometry} material={materials.band} material-color={currentState.items.band} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_7.geometry} material={materials.patch} material-color={currentState.items.patch} />
      </group>
    </>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')

Open BlenderScene.js file and copy the following code
We add state in order to know which part of the model was selected
Added work with the picker component
Added animation to the model, floating illusion

import React, { useState, useEffect, Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import { HexColorPicker } from 'react-colorful'
import Model from './ShoeModelDraco'
function Scene() {
  // Cursor showing current color
  const [state, setState] = useState({
    current: null,
    items: {
      laces: "#ffffff",
      mesh: "#ffffff",
      caps: "#ffffff",
      inner: "#ffffff",
      sole: "#ffffff",
      stripes: "#ffffff",
      band: "#ffffff",
      patch: "#ffffff",
    },
  });
  const [hovered, setHover] = useState(null)
  useEffect(() => {
    const cursor = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><g clip-path="url(#clip0)"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><g filter="url(#filter0_d)"><path d="M29.5 47C39.165 47 47 39.165 47 29.5S39.165 12 29.5 12 12 19.835 12 29.5 19.835 47 29.5 47z" fill="${state.items[hovered]}"/></g><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/><text fill="#000" style="white-space:pre" font-family="Inter var, sans-serif" font-size="10" letter-spacing="-.01em"><tspan x="35" y="63">${hovered}</tspan></text></g><defs><clipPath id="clip0"><path fill="#fff" d="M0 0h64v64H0z"/></clipPath><filter id="filter0_d" x="6" y="8" width="47" height="47" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feColorMatrix in="SourceAlpha" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0"/><feOffset dy="2"/><feGaussianBlur stdDeviation="3"/><feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0"/><feBlend in2="BackgroundImageFix" result="effect1_dropShadow"/><feBlend in="SourceGraphic" in2="effect1_dropShadow" result="shape"/></filter></defs></svg>`
    const auto = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/></svg>`
    if (hovered) {
      document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(cursor)}'), auto`
      return () => (document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(auto)}'), auto`)
    }
  }, [hovered])

  function Picker() {
    return (
      <div style={
        {
          display: state.current ? "block" : "none",
          position: "absolute",
          top: "50px",
          left: "50px",

       }
      }>
        <HexColorPicker
          className="picker"
          color={state.items[state.current]}
          onChange={(color) => {
            let items = state.items;
            items[state.current] =  color
          }}
        />
        <h1>{state.current}</h1>
      </div>
    )
  }
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model
            currentState={ state }
            setCurrentState={(curState) => {
              setState({
                ...state,
                current: curState
              })
            }}
            setHover={ setHover}
          />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
      <Picker />
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>

  );
}

export default BlenderScene;

If everything works successfully, you should see something like this:
In the resources part, you can find a link for the branch with the code until this step
Live Working example here

Resources

Appendix

Blender
- Blender is the free and open-source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing, and motion tracking, even video editing, and game creation; more in here
glTF files
- Graphics Language Transmission Format or GL Transmission Format, more here
gltf-pipeline
- Content pipeline tools for optimizing glTF, more here

DEV Community

From Desktop 3d Apps to Web 3d Apps using Blender and React

Before you start

What is Blender? Simply Explained

Create React App

Install dependencies

Export blender asset

Compress asset

Convert asset to `JSX` component

Integrate new component

Enhanced component and functionality

Adding some style

Install dependency

Edit React Components

Resources

Appendix

Top comments (0)

Read next

Self Writing Lang Graph State

GraphQL: A Beginner's Guide

TypeScript for Domain-Driven Design (DDD)

.NET Development and Localization for JustAnswer – case study

Before you start

What is Blender? Simply Explained

Create React App

Install dependencies

Export blender asset

Compress asset

Convert asset to JSX component

Integrate new component

Enhanced component and functionality

Adding some style

Install dependency

Edit React Components

Resources

Appendix

Read next

Self Writing Lang Graph State

GraphQL: A Beginner's Guide

TypeScript for Domain-Driven Design (DDD)

.NET Development and Localization for JustAnswer – case study

Convert asset to `JSX` component