How to click and drag 3D models in ThreeJS

Having the ability to move 3D models around in a three.js project can have nearly endless applications...

Models by: Stanley Creative, Johnson Martin, and Gravity Jack

...this article will explain how to go about adding this functionality to your app. While also addressing the complexities that arise when these 3D models are comprised of multiple objects themselves.

This article is broken up into 2 sections:

    A) Moving individual objects (Object3D)
    B) Moving objects with children (Group)

Section "A" will lay the groundwork. It will introduce concepts and functions that will be needed when moving a 3D model. If you already have a good understanding of moving Object3Ds in three.js you can either skim through this section or simply skip it all together, and head directly to the next section. Section "B" will dive into how to actually move a 3D model in three.js and the complexities of an Object3D having children.

A) Moving individual objects (Object3D)

Singular objects in three.js are handled by the Objects3D class.
Every single object in a scene will always be its own Object3D.

Some examples of this are the built in geometry shapes that can easily be added to a scene. These singular objects come in a wide range of shapes, each having several options for customization.

This section will show how to add these Object3Ds to a scene and then how to move them around using mouse controls.

(A live demo of this section's code can be found here: Move-Object3D.)

1) Create and setup Scene

We will need a three.js Scene with a Camera, Lights, Renderer, Controls, and any other desired attributes. Here is a basic template that you can build off of, if you don't already have one.

2) Add the object

For this example, we are going to create a cylinder, but this could easily be any basic shape that three.js provides. Here is the code to do this:

function addObject(radius, pos, color) {
  const object = new THREE.Mesh(
    new THREE.CylinderBufferGeometry(radius, radius, 10, 50),
    new THREE.MeshPhongMaterial({ color: color })
  );
  object.position.set(pos.x, pos.y, pos.z);
  object.isDraggable = true;
  scene.add(object);
};

As you can see the const object is the variable that the cylinder Object3D is stored into. The size, color and detail are fully customizable and don't have to match what is shown.

From here we just set a few basic properties.
The position is a default property with a built in set() function and the isDraggable is a custom property that was added for use later on.
Once we set the desired properties all we do is simply add it to the scene like so...

addObject(8, { x: 0, y: 6, z: 0 }, '#FF0000');

3) Holding variable for the object

We might have multiple objects in a scene; however, we only want to move one at a time. One easy approach to this is to create a container variable that will hold the object we want to move; we can then manipulate this container on a global scale without each of our functions having to know which specific object was chosen. The function instead will just make general changes to the container that will ‘trickle’ down to the object. We will see this in action in the next step.
For this example, I’ve named this container draggableObject.

// Global variables
Let draggableObject;

4) Mouse click event listener

To select an object we will need to be able to have a listener to track mouse clicks.

window.addEventListener('click', event => {
  // If 'holding' object on-click, set container to <undefined> to 'drop’ the object.
  if (draggableObject) {
    draggableObject= undefined;
    return;
  }

  // If NOT 'holding' object on-click, set container to <object> to 'pick up' the object.
  clickMouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  clickMouse.y = - (event.clientY / window.innerHeight) * 2 + 1;
  raycaster.setFromCamera(clickMouse, camera);
  const found = raycaster.intersectObjects(scene.children, true);
  if (found.length && found[0].object.isDraggable) {
    draggableObject = found[0].object;
  }
});

Okay, there is a lot going on here so let’s break this down.
Firstly, we need to understand how our object is going to be moved. For this tutorial on the first click we will pick up the object. Once holding an object we can then move our mouse anywhere within the window to move the object. Then on a second click we will “drop” the object.

With this understanding let’s look at the code. The first short circuit if-statement is to handle the drop. If we aren’t holding an object then we continue to determine which object to pick up on the click (if there is any valid object).

To find an object we are using a raycaster. The way this works is it creates a line starting from the Camera’s position and travels to the mouse click location and then continues through all objects until it reaches the end of the scene. Because of this we need to get the x and y locations of the mouse click to be able to create this line.

Finally, this raycaster returns an array of all the objects it passed through and an empty array if it passed through no objects. To determine which object we want to move we need to check two things. Are there any objects found? found.length and is the first object in the array draggable? found[0].object.isDraggable. (This is where that custom property from step 1 comes into play). If you have a floor, walls, ceiling, or other objects you don’t want to be draggable, you can simply make this Boolean false and the function ends here.

Now that we've gotten to the end of the function and have found a valid object to move, we need to store it into the container variable draggableObject. We can now edit the position of this container in another function.

5) Mouse move event listener

Before we can move the container we need to be able to track the mouse's position. This basic listener will do just that. With this information we can re-render the object as we move it along the mouse's path.

window.addEventListener('mousemove', event => {
  moveMouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  moveMouse.y = - (event.clientY / window.innerHeight) * 2 + 1;
});

6) Create a function to drag the object

We’re almost done. We just need a function to allow us to move the selected object that resides in the draggableObject. This function can utilize the mouse move listener we just created.

function dragObject() {
  // If 'holding' an object, move the object
  if (draggableObject) {
  const found = raycaster.intersectObjects(scene.children);
  // `found` is the metadata of the objects, not the objetcs themsevles  
    if (found.length) {
      for (let obj3d of found) {
        if (!obj3d.object.isDraggablee) {
          draggableObject.position.x = obj3d.point.x;
          draggableObject.position.z = obj3d.point.z;
          break;
        }
      }
    }
  }
};

As you can see the first thing we are doing is checking if the container is empty (undefined) or if it contains an object. If it does contain an object, we need to be able to move it across the floor.

We create another raycaster to check all intersections, and if the ground is still under the object we want to move. Basically, it is tracking the mouse movement with moveMouse and finding where the mouse location is intersecting with other objects (in this case the floor with the isDraggablee = false). It then updates the containers’ position with these results which in turn, update the object within it.

This is great and exactly what we want, but for this function to work it needs to be continuously called. Otherwise we won't have a live representation of the object being dragged around. The solution to this is actually super simple. All we need to do is place this function inside the mouse listener event, like so…

window.addEventListener('mousemove', event => {
  dragObject();
  moveMouse.x = (event.clientX / window.innerWidth) * 2 - 1;
  moveMouse.y = - (event.clientY / window.innerHeight) * 2 + 1;
});

And with this we are done, we can now pick up, hold, and drop any objects in the scene. Congrats!

B) Moving objects with children (Group)

Model by: Stanley Creative

This section will replace the default geometry object that three.js provides, with a 3D model of our own choosing. In the case of this example, it'll be from a local directory source.

The very important thing to note is that a 3D Model is not a single Object3D like the shapes from the section above are. Instead they are Groups with multiple Object3D children. Even the simplest of models will have some complexity to them. This is why this section is so important.

(A live demo of this section's code can be found here: Move-Group.)

1) Setup and Create scene

Make sure you have the fundamentals of the three.js app already in place. _Head back to Section A or visit the live demo if you don't have anything created yet.

2) Add the Model

Similar to the addObject() function we need one that will be able to load our assets into the scene we have created.

function addModel(pos) {
  const loader = new GLTFLoader();
  loader.load(`res/saturnV/scene.gltf`, (gltf) => {
    const model = gltf.scene;
    model.position.set(pos.x, pos.y, pos.z);
    model.isDraggable = true;
    scene.add(model);
  });
}

The first thing to notice is this function utilizes the GLTFLoader. Make sure you have this imported in some way into your program. You can check here for installation instructions or checkout how I did it in the demo.

With the loader we are simply just telling it where to load the files from. In the case of this example, they reside in a directory within the res folder.

Once the const model is populated we edit the properties; making sure we include the isDraggable = true and add it to the scene just as we did for the object in section A.

Once this is created we just need this function...

addModel({ x: 0, y: 6, z: 0 });

3) Objects3D vs Groups

THE PROBLEM:
If you try to test your app at this current stage it most likely will not work. There are two problems that you might face.

1. Your model is never picked up and thus you cannot move it at all.
2. You can only move a single piece of the model at a time. Resulting in you ripping it apart piece by piece.

Model by: Stanley Creative

Why is this?
The reason for these undesired results is caused by how the model assets are saved, and how the GLTFLoader loads them into the scene.

Unlike the simple Object3D, models are usually comprised of multiple Object3Ds; sometimes even hundreds. Because of this, the GLTFLoader puts all these Object3Ds into a Group. These Groups work almost identical to the Object3Ds except for the obvious fact that they are groups.

NOTE: Even the most basic of models that happens to be a single Object3D (extremely rare this happens). It will still be loaded in as a Group. Simply a Group with a single Object3D.

This all means when we set const model = gltf.scene; in the prior step, we weren’t setting an Object3D to the const model but a Group. Thus, our group is now draggable but the individual objects within the group are not. To top this off, currently our raycaster is only looking for an Object3D and not Groups.

THE SOLUTION:
To fix this issue the best solution is to change what we put into the draggable container. We need to place the entire group into the container.

To do this we need to understand that Groups are structured as Trees. Each Object3D within a Group can have none to multiple children. Because of this it can get complicated if we try to access every single node, thus we aren’t going to do that. Instead, we are just going to select an Object3D (any of them) within the Group when we click, and then traverse through each parent until we reach the top. This top layer will be the Group created by the GLTFLoader with the isDraggable = true.

To do this we are going to take the addEventListener(‘click’, event… from Section A step 4 above and change the if-statement after the raycaster finds an object.
This is what the code will look like...

const found = raycaster.intersectObjects(scene.children, true);
  if (found.length) {
  // Cycle upwards through every parent until it reaches the topmost layer (the Group)
  let current = found[0].object;
  while (current.parent.parent !== null) {
    current = current.parent;
  }
  if (current.isDraggable) {
    draggableModel = current;
  }
}

With this set up, it doesn’t matter how many nodes are in the Group tree, eventually we will reach the top layer. Once here we check for the isDraggable Boolean. If it's true we can now pick up the model and move it just as before.

It's good to note that even though we changed this, the code here will still allow us to pick up Groups with a single Object3D, as well as Object3D that aren't in a Groups at all.

4) Conclusion

And with that we're all done.
We can now load our models into a scene and move them around regardless of how complicated the models are. We can also move around the built-in shapes at the same time.

The complete repo for all the code above can be found here
The live demos can be found here:
Move-Object3D
Move-Groups

Thanks for reading.
If you have any questions or comments please feel free to reach out to me.
My information: GitHub, Linkedin