Recently I'm working on handtracking ARs. To align 3D models to hands, I had to match coordinates of 3D model objects to a coordinate of a hand to make effects like 3D models on a hand etc.

It is a bit complicated to align them, so let me explain how to do it here.

Assumptions

Let's say we have a target coordinate XYZ (e.g. hand) and a coordinate xyz (e.g. 3D model) to align. We already know the vectors of XYZ and xyz. X/Y/Z are orthogonal each other and normalized (the length is 1) and x/y/z are also. The goal is to align the xyz vectors to XYZ.

Aligning the origin positions are not explained in this article because we can do it just by setting the XYZ origin position to xyz origin position.

In the following steps, I align z/Z and x/X. However, the order of operation does not matter. You may align y/Y first and z/Z next.

Align z to Z

First, let's align the z axis.

Calculate the cross vector z × Z and θ with the dot product and Arccosine which is used to rotate the coordinate around the cross vector. To achieve the rotation, we can use a quaternion. Calculating quaternion by ourselves is a bit complicated and usually 3D CG engine has built-in function to calculate it from the axis to rotate around and the angle. If you really want to implement by yourself, you may refer this article.

This is an example pseudocode. Your programming language / 3D CG engine (e.g. Three.js / Unity / Unreal Engine) should have cross / dot / normalize / acos / fromAxisAngle.

Vec3 z = ... // Your z
Vec3 Z = ... // Your Z
Vec3 axisZ = normalize(cross(z, Z)); // Normalized cross vector.
float angleZ = acos(dot(z, Z)); // The angle between z and Z --> theta in the image.
Quaternion qz = fromAxisAngle(axisZ, angleZ); // Later we will make other quaternions so let's name it qz.

Now we've got the quaternion qz to align z to Z axis.

Align x to X

By applying the quaternion qz to your coordinate, you should have x'/y'/z' like the following image.

Aligning x/X axes is basically the same process as we did for z/Z. Since z/Z have been aligned already, x and X is on the same plane. When x/X is aligned, y/Y is automatically aligned. However, please do not forget apply the quaternion qz to the x vector. Let's name the rotated x vector as x'.

Pseudocode should be like this. applyQuaternion is a function to apply a quaternion to a vector. Your programming language / 3D engine should have an equivalent.

Vec3 rotatedX = applyQuaternion(x, qz); // x'
Vec3 X = ... // Your X
Vec3 axisX = normalize(cross(rotatedX, X));
float angleX = acos(dot(rotatedX, X)); // The angle between x' and X --> phi in the image
Quaternion qx = fromAxisAngle(axisX, angleX);

We've got the next quaternion qx.

Combine quaternions

Now we have qz and qx. By combining the quaternions, we will have the final quaternion we can realize our coordinate alignment.

multiplyQuaternions is a function to combine two quaternions into one. Your programming language / 3D engine should have an equivalent. Please be careful that the order of qz and qx is important.

Quaternion q = mltiplyQuaternions(qx, qz);

By applying the quaternion q to x/y/z respectively, you'll get vectors identical to X/Y/Z.

Appendix: Three.js code

I showed pseudo code in the above. I also give you guys an example in Three.js.

// Your coordinates.
// x/y/z are vectors that will be aligned FROM.
const x = new Vector3(...);
const y = new Vector3(...);
const z = new Vector3(...); // <-- You do not need this actually.

// X/Y/Z are vectors that will be aligned TO.
const X = new Vector3(...);
const Y = new Vector3(...);
const Z = new Vector3(...); // <-- You do not need this actually.


const axisZ = z.clone().cross(Z).normalize();
const angleZ = Math.acos(z.dot(Z));
const qz = new Quaternion().setFromAxisAngle(axisZ, angleZ);

const rotatedX = x.clone().applyQuaternion(qz);
const axisX = rotatedX.clone().cross(X).normalize();
const angleX = Math.acos(rotatedX.dot(X));
const qx = new Quaternion().setFromAxisAngle(axisX, angleX);

// Final quaternion
const q = new Quaternion().multiplyQuaternions(qx, qz);

// You can use q like this.
yourModel.rotation.setFromQuaternion(q)

You can check my Demo, if you want!