برچسب: Objects

When building the basement studio site, we wanted to add 3D characters without compromising performance. We used instancing to render all the characters simultaneously. This post introduces instances and how to use them with React Three Fiber.

Introduction

Instancing is a performance optimization that lets you render many objects that share the same geometry and material simultaneously. If you have to render a forest, you’d need tons of trees, rocks, and grass. If they share the same base mesh and material, you can render all of them in a single draw call.

A draw call is a command from the CPU to the GPU to draw something, like a mesh. Each unique geometry or material usually needs its own call. Too many draw calls hurt performance. Instancing reduces that by batching many copies into one.

Basic instancing

As an example, let’s start by rendering a thousand boxes in a traditional way, and let’s loop over an array and generate some random boxes:

const boxCount = 1000

function Scene() {
  return (
    <>
      {Array.from({ length: boxCount }).map((_, index) => (
        <mesh
          key={index}
          position={getRandomPosition()}
          scale={getRandomScale()}
        >
          <boxGeometry />
          <meshBasicMaterial color={getRandomColor()} />
        </mesh>
      ))}
    </>
  )
}

If we add a performance monitor to it, we’ll notice that the number of “calls” matches our boxCount.

A quick way to implement instances in our project is to use drei/instances.

The Instances component acts as a provider; it needs a geometry and materials as children that will be used each time we add an instance to our scene.

The Instance component will place one of those instances in a particular position/rotation/scale. Every Instance will be rendered simultaneously, using the geometry and material configured on the provider.

import { Instance, Instances } from "@react-three/drei"

const boxCount = 1000

function Scene() {
  return (
    <Instances limit={boxCount}>
      <boxGeometry />
      <meshBasicMaterial />
      {Array.from({ length: boxCount }).map((_, index) => (
        <Instance
          key={index}
          position={getRandomPosition()}
          scale={getRandomScale()}
          color={getRandomColor()}
        />
      ))}
    </Instances>
  )
}

Notice how “calls” is now reduced to 1, even though we are showing a thousand boxes.

What is happening here? We are sending the geometry of our box and the material just once to the GPU, and ordering that it should reuse the same data a thousand times, so all boxes are drawn simultaneously.

Notice that we can have multiple colors even though they use the same material because Three.js supports this. However, other properties, like the map, should be the same because all instances share the exact same material.

We’ll see how we can hack Three.js to support multiple maps later in the article.

Having multiple sets of instances

If we are rendering a forest, we may need different instances, one for trees, another for rocks, and one for grass. However, the example from before only supports one instance in its provider. How can we handle that?

The creteInstnace() function from drei allows us to create multiple instances. It returns two React components, the first one a provider that will set up our instance, the second, a component that we can use to position one instance in our scene.

Let’s see how we can set up a provider first:

import { createInstances } from "@react-three/drei"

const boxCount = 1000
const sphereCount = 1000

const [CubeInstances, Cube] = createInstances()
const [SphereInstances, Sphere] = createInstances()

function InstancesProvider({ children }: { children: React.ReactNode }) {
  return (
    <CubeInstances limit={boxCount}>
      <boxGeometry />
      <meshBasicMaterial />
      <SphereInstances limit={sphereCount}>
        <sphereGeometry />
        <meshBasicMaterial />
        {children}
      </SphereInstances>
    </CubeInstances>
  )
}

Once we have our instance provider, we can add lots of Cubes and Spheres to our scene:

function Scene() {
  return (
    <InstancesProvider>
      {Array.from({ length: boxCount }).map((_, index) => (
        <Cube
          key={index}
          position={getRandomPosition()}
          color={getRandomColor()}
          scale={getRandomScale()}
        />
      ))}

      {Array.from({ length: sphereCount }).map((_, index) => (
        <Sphere
          key={index}
          position={getRandomPosition()}
          color={getRandomColor()}
          scale={getRandomScale()}
        />
      ))}
    </InstancesProvider>
  )
}

Notice how even though we are rendering two thousand objects, we are just running two draw calls on our GPU.

Instances with custom shaders

Until now, all the examples have used Three.js’ built-in materials to add our meshes to the scene, but sometimes we need to create our own materials. How can we add support for instances to our shaders?

Let’s first set up a very basic shader material:

import * as THREE from "three"

const baseMaterial = new THREE.RawShaderMaterial({
  vertexShader: /*glsl*/ `
    attribute vec3 position;
    attribute vec3 instanceColor;
    attribute vec3 normal;
    attribute vec2 uv;
    uniform mat4 modelMatrix;
    uniform mat4 viewMatrix;
    uniform mat4 projectionMatrix;

    void main() {
      vec4 modelPosition = modelMatrix * vec4(position, 1.0);
      vec4 viewPosition = viewMatrix * modelPosition;
      vec4 projectionPosition = projectionMatrix * viewPosition;
      gl_Position = projectionPosition;
    }
  `,
  fragmentShader: /*glsl*/ `
    void main() {
      gl_FragColor = vec4(1, 0, 0, 1);
    }
  `
})

export function Scene() {
  return (
    <mesh material={baseMaterial}>
      <sphereGeometry />
    </mesh>
  )
}

Now that we have our testing object in place, let’s add some movement to the vertices:

We’ll add some movement on the X axis using a time and amplitude uniform and use it to create a blob shape:

const baseMaterial = new THREE.RawShaderMaterial({
  // some unifroms
  uniforms: {
    uTime: { value: 0 },
    uAmplitude: { value: 1 },
  },
  vertexShader: /*glsl*/ `
    attribute vec3 position;
    attribute vec3 instanceColor;
    attribute vec3 normal;
    attribute vec2 uv;
    uniform mat4 modelMatrix;
    uniform mat4 viewMatrix;
    uniform mat4 projectionMatrix;

    // Added this code to shift the vertices
    uniform float uTime;
    uniform float uAmplitude;
    vec3 movement(vec3 position) {
      vec3 pos = position;
      pos.x += sin(position.y + uTime) * uAmplitude;
      return pos;
    }

    void main() {
      vec3 blobShift = movement(position);
      vec4 modelPosition = modelMatrix * vec4(blobShift, 1.0);
      vec4 viewPosition = viewMatrix * modelPosition;
      vec4 projectionPosition = projectionMatrix * viewPosition;
      gl_Position = projectionPosition;
    }
  `,
  fragmentShader: /*glsl*/ `
    void main() {
      gl_FragColor = vec4(1, 0, 0, 1);
    }
  `,
});

export function Scene() {
  useFrame((state) => {
    // update the time uniform
    baseMaterial.uniforms.uTime.value = state.clock.elapsedTime;
  });

  return (
    <mesh material={baseMaterial}>
      <sphereGeometry args={[1, 32, 32]} />
    </mesh>
  );
}

Now, we can see the sphere moving around like a blob:

Now, let’s render a thousand blobs using instancing. First, we need to add the instance provider to our scene:

import { createInstances } from '@react-three/drei';

const [BlobInstances, Blob] = createInstances();

function Scene() {
  useFrame((state) => {
    baseMaterial.uniforms.uTime.value = state.clock.elapsedTime;
  });

  return (
    <BlobInstances material={baseMaterial} limit={sphereCount}>
      <sphereGeometry args={[1, 32, 32]} />
      {Array.from({ length: sphereCount }).map((_, index) => (
        <Blob key={index} position={getRandomPosition()} />
      ))}
    </BlobInstances>
  );
}

The code runs successfully, but all spheres are in the same place, even though we added different positions.

This is happening because when we calculated the position of each vertex in the vertexShader, we returned the same position for all vertices, all these attributes are the same for all spheres, so they end up in the same spot:

vec3 blobShift = movement(position);
vec4 modelPosition = modelMatrix * vec4(deformedPosition, 1.0);
vec4 viewPosition = viewMatrix * modelPosition;
vec4 projectionPosition = projectionMatrix * viewPosition;
gl_Position = projectionPosition;

To solve this issue, we need to use a new attribute called instanceMatrix. This attribute will be different for each instance that we are rendering.

  attribute vec3 position;
  attribute vec3 instanceColor;
  attribute vec3 normal;
  attribute vec2 uv;
  uniform mat4 modelMatrix;
  uniform mat4 viewMatrix;
  uniform mat4 projectionMatrix;
  // this attribute will change for each instance
  attribute mat4 instanceMatrix;

  uniform float uTime;
  uniform float uAmplitude;

  vec3 movement(vec3 position) {
    vec3 pos = position;
    pos.x += sin(position.y + uTime) * uAmplitude;
    return pos;
  }

  void main() {
    vec3 blobShift = movement(position);
    // we can use it to transform the position of the model
    vec4 modelPosition = instanceMatrix * modelMatrix * vec4(blobShift, 1.0);
    vec4 viewPosition = viewMatrix * modelPosition;
    vec4 projectionPosition = projectionMatrix * viewPosition;
    gl_Position = projectionPosition;
  }

Now that we have used the instanceMatrix attribute, each blob is in its corresponding position, rotation, and scale.

Changing attributes per instance

We managed to render all the blobs in different positions, but since the uniforms are shared across all instances, they all end up having the same animation.

To solve this issue, we need a way to provide custom information for each instance. We actually did this before, when we used the instanceMatrix to move each instance to its corresponding location. Let’s debug the magic behind instanceMatrix, so we can learn how we can create own instanced attributes.

Taking a look at the implementation of instancedMatrix we can see that it is using something called InstancedAttribute:

InstancedBufferAttribute allows us to create variables that will change for each instance. Let’s use it to vary the animation of our blobs.

Drei has a component to simplify this called InstancedAttribute that allows us to define custom attributes easily.

// Tell typescript about our custom attribute
const [BlobInstances, Blob] = createInstances<{ timeShift: number }>()

function Scene() {
  useFrame((state) => {
    baseMaterial.uniforms.uTime.value = state.clock.elapsedTime
  })

  return (
    <BlobInstances material={baseMaterial} limit={sphereCount}>
      {/* Declare an instanced attribute with a default value */}
      <InstancedAttribute name="timeShift" defaultValue={0} />
      
      <sphereGeometry args={[1, 32, 32]} />
      {Array.from({ length: sphereCount }).map((_, index) => (
        <Blob
          key={index}
          position={getRandomPosition()}
          
          // Set the instanced attribute value for this instance
          timeShift={Math.random() * 10}
          
        />
      ))}
    </BlobInstances>
  )
}

We’ll use this time shift attribute in our shader material to change the blob animation:

uniform float uTime;
uniform float uAmplitude;
// custom instanced attribute
attribute float timeShift;

vec3 movement(vec3 position) {
  vec3 pos = position;
  pos.x += sin(position.y + uTime + timeShift) * uAmplitude;
  return pos;
}

Now, each blob has its own animation:

Creating a forest

Let’s create a forest using instanced meshes. I’m going to use a 3D model from SketchFab: Stylized Pine Tree Tree by Batuhan13.

import { useGLTF } from "@react-three/drei"
import * as THREE from "three"
import { GLTF } from "three/examples/jsm/Addons.js"

// I always like to type the models so that they are safer to work with
interface TreeGltf extends GLTF {
  nodes: {
    tree_low001_StylizedTree_0: THREE.Mesh<
      THREE.BufferGeometry,
      THREE.MeshStandardMaterial
    >
  }
}

function Scene() {

  // Load the model
  const { nodes } = useGLTF(
    "/stylized_pine_tree_tree.glb"
  ) as unknown as TreeGltf

  return (
    <group>
      {/* add one tree to our scene */ }
      <mesh
        scale={0.02}
        geometry={nodes.tree_low001_StylizedTree_0.geometry}
        material={nodes.tree_low001_StylizedTree_0.material}
      />
    </group>
  )
}

(I added lights and a ground in a separate file.)

Now that we have one tree, let’s apply instancing.

const getRandomPosition = () => {
  return [
    (Math.random() - 0.5) * 10000,
    0,
    (Math.random() - 0.5) * 10000
  ] as const
}

const [TreeInstances, Tree] = createInstances()
const treeCount = 1000

function Scene() {
  const { scene, nodes } = useGLTF(
    "/stylized_pine_tree_tree.glb"
  ) as unknown as TreeGltf

  return (
    <group>
      <TreeInstances
        limit={treeCount}
        scale={0.02}
        geometry={nodes.tree_low001_StylizedTree_0.geometry}
        material={nodes.tree_low001_StylizedTree_0.material}
      >
        {Array.from({ length: treeCount }).map((_, index) => (
          <Tree key={index} position={getRandomPosition()} />
        ))}
      </TreeInstances>
    </group>
  )
}

Our entire forest is being rendered in only three draw calls: one for the skybox, another one for the ground plane, and a third one with all the trees.

To make things more interesting, we can vary the height and rotation of each tree:

const getRandomPosition = () => {
  return [
    (Math.random() - 0.5) * 10000,
    0,
    (Math.random() - 0.5) * 10000
  ] as const
}

function getRandomScale() {
  return Math.random() * 0.7 + 0.5
}

// ...
<Tree
  key={index}
  position={getRandomPosition()}
  scale={getRandomScale()}
  rotation-y={Math.random() * Math.PI * 2}
/>
// ...

Further reading

There are some topics that I didn’t cover in this article, but I think they are worth mentioning:

• Batched Meshes: Now, we can render one geometry multiple times, but using a batched mesh will allow you to render different geometries at the same time, sharing the same material. This way, you are not limited to rendering one tree geometry; you can vary the shape of each one.
• Skeletons: They are not currently supported with instancing, to create the latest basement.studio site we managed to hack our own implementation, I invite you to read our implementation there.
• Morphing with batched mesh: Morphing is supported with instances but not with batched meshes. If you want to implement it yourself, I’d suggest you read these notes.