I've been looking into creating a texture from data in three.js. It is super easy, but there are some caveats, and some parts can be confusing. I fell into some traps many years ago, then fall into it again recently, so I decided to write about it!

• What is confusing (me)?
• Grayscale textures
  • Which format
  • Luminance format allows 3 types (WebGL1)
  • Type to TypedArray containing the data
  • Access the data in the fragment shader
• Gimme some concrete examples!
  • UnsignedByte Texture
  • HalfFloat Texture
  • Float Texture
• Until next time!

Table of contents generated with markdown-toc

What is confusing (me)?

When creating a new texture, from data, you must set a Format, a Type, and provide you data in a specific type of TypedArray.

const texture = new DataTexture(data, width, height, format, type, ...);

The doc says

"type" must correspond to the "format".

Ok... so how do I know which Type to set for the Format I want to use?

Grayscale textures

In this post, I will only discuss grayscale (single channel) texture for WebGL1 since it is my current focus. Everything in the rest of this post will apply to whatever you are trying to support.

Which format

Alright, I want to create a single channel (grayscale) texture.
In the 'internal format' section, you can find which Format is the best fit for you, depending on the number of channels and the bytes per pixel.

LUMINANCE is the natural fit for grayscale textures. The caveat is that it only supports UnsignedByte texture type for Webgl1.

If you work with WebGL2, R* formats allow you to support a variety of different bit-depth.

To know more about what the different file format means, I found this page useful: https://www.khronos.org/opengl/wiki/Image_Format. It explains what the suffix of the file format means *F, *_SNORM means and how those type of textures are interpreted. That is important regarding the data normalization. (keep reading)

Luminance format allows three types (WebGL1)

Alright, we just learned that for WebGL1, Luminance format goes with UnsignedByte type.

Can we do better?

If your browser supports the OES_texture_float extension, a bunch of new types (Float and HalfFloat) are available for the LUMINANCE format. (official documentation)

Type to TypedArray containing the data

It is pretty straight forward:

What is important there is that the number of bits in the type array matches the byte per pixel in the table. Also, for HalfFloat the data should be prepared appropriately.

Access the data in the fragment shader

All the integer textures (including UnsignedByteType) are normalized automatically while uploaded to the shaders, whereas the floating/integral textures (including Float and HalfFloat) are passed as it is.

Based on the Format name, you can know with which type of data you are dealing with and whether that will be normalized for you or not. (ref).

In other words, in the fragment shader, when using an UnsignedByteType texture, the values you get from the texture 2D are normalized between 0 and 1 automatically. For FloatType and HalfFloatType you get the value that was in the typed array without any normalization.

Gimme some concrete examples!

UnsignedByte Texture

const textureSize = 16
const dataSize = 10;
const data = new Uint8Array(dataSize);

for (let i = 0; i < dataSize) {
  data[i] = Math.round(Math.random() * 255); // pass anything from 0 to 255
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, UnsignedByteType);

varying vec2 vUv;
uniform sampler2D uData;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  gl_FragColor = vec4( data.xyz, 1.0 );
}

HalfFloat Texture

To convert a number to half float, do it the three.js way: like this

⚠️ Watch out for precision errors when converting numbers to half float precision!

const textureSize = 16
const dataSize = 10;
const data = new Uint16Array(dataSize);

for (let i = 0; i < dataSize) {
  const largeNumber = Math.random() * 10000; // pass anything from 0 to 10000
  data[i] = toHalfFloat(largeNumber);
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, HalfFloatType);

varying vec2 vUv;
uniform sampler2D uData;
uniform float uMax;
uniform float uMin;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  vec4 normalizedData = (data - uMin) / (uMax - uMin);
  gl_FragColor = vec4( data.xyz, 1.0 );
}

Float Texture

const textureSize = 16
const dataSize = 10;
const data = new Float32Array(dataSize);

for (let i = 0; i < dataSize) {
  const largeNumber = Math.random() * 10000; // pass anything from 0 to 10000
  data[i] = largeNumber;
}

const texture = new DataTexture(data, textureSize, textureSize, LUMINACE, FloatType);

varying vec2 vUv;
uniform sampler2D uData;
uniform float uMax;
uniform float uMin;

void main(){
  vec3 color;
  vec4 data = texture2D( uData, vUv );
  vec4 normalizedData = (data - uMin) / (uMax - uMin);
  gl_FragColor = vec4( data.xyz, 1.0 );
}

Until next time!

Until next time 🙋‍♂️, happy coding!