The #1Millioncubes challenge. How I managed to animate a freakload of cubes on windows using Unity

20/03/2023 update

I wrote a new demo that doesn’t need to use GPUInstancer. Code is now available on github. Note that I also updated the code to use GraphicBuffers instead than ComputeBuffers.

For the game I am currently developing, Gamecraft, we are using Svelto.ECS for the game logic, Unity ECS for the physic and GPUInstancer compute shader pipeline for the rendering. I have never had the time to benchmark the UECS rendering pipeline against the GPUInstancer one, mainly because I decided directly to use GPUInstancer, as its performance even on low-end GPUs (read intel HD4000) is astonishing.

Recently, on twitter, I have been asked if it was feasible to move 1 million 3D objects. I already knew that GPUInstancer wouldn’t break a sweat, but I also knew from my previous experiments that uploading from the CPU to the GPU 1 million matrices would have been impossible to use practically.

With GPUInstancer you are not supposed to update ALL the objects, but only the ones that actually moves, which usually are less. However knowing what the bottleneck is, I wondered if it was possible to upload the data to the GPU asynchronously or in any other way.

It turns out that Unity 2020 introduced two new ComputeBuffer methods, called BeginWrite and EndWrite. This sounded exactly what I needed to make this experiment work!

Let’s see what happens before and after using these methods. This is the entire code used to create the demo as seen in the tweet:

//#define OLD_WAY

using System;
using GPUInstancer;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Jobs;
using Unity.Mathematics;
using Unity.Profiling;
using UnityEngine;

public class Bootstrap : MonoBehaviour
{
    static readonly int numberOfObjectsPerSide = 1000;
    static readonly int totalObjects           = numberOfObjectsPerSide * numberOfObjectsPerSide;

    // Start is called before the first frame update
    void Start()
    {
        _manager = FindObjectOfType<GPUInstancerPrefabManager>();

        var prototypeGameObject = Resources.Load<GameObject>("Cube");

        _prefabPrototype = _manager.DefineGameObjectAsPrefabPrototypeAtRuntime(prototypeGameObject);

        var matrices = new Matrix4x4[totalObjects];

        GPUInstancerAPI.InitializeWithMatrix4x4Array(_manager, _prefabPrototype, matrices);

        _transformationMatrixVisibilityBuffer =
            _manager.GetRuntimeData(_prefabPrototype, true).transformationMatrixVisibilityBuffer;
        
        //not that nice, but it's a quick way to be able to do the EndWrite at the begin og the Update()
        var array = _transformationMatrixVisibilityBuffer.BeginWrite<Matrix4x4>(0, totalObjects);
        
        for (var i = numberOfObjectsPerSide - 1; i >= 0; --i)
        for (var j = numberOfObjectsPerSide - 1; j >= 0; --j)
            array[i + j * numberOfObjectsPerSide] =
                float4x4.TRS(new float3(i * 1.5f, 0, j * 1.5f), quaternion.identity, new float3(1, 1, 1));

        _then = DateTime.Now;

#if OLD_WAY
        _array = new NativeArray<Matrix4x4>(matrices, Allocator.Persistent);
#endif
    }

#if OLD_WAY
    void OnDestroy() { _array.Dispose(); }
#endif

    // Update is called once per frame
    void Update()
    {
        _jobhandle.Complete();

#if OLD_WAY
        var profilerMarker = new ProfilerMarker("SetData");
        profilerMarker.Begin();
        _transformationMatrixVisibilityBuffer.SetData(_array);
        profilerMarker.End();
#else
        var profilerMarker = new ProfilerMarker("SetData");
        profilerMarker.Begin();
        //now that we are sure that the job is complete, we can do end write and start a new write.
        //not sure if this is actually necessary
        _transformationMatrixVisibilityBuffer.EndWrite<Matrix4x4>(totalObjects);
        _array = _transformationMatrixVisibilityBuffer.BeginWrite<Matrix4x4>(0, totalObjects);
        profilerMarker.End();
#endif
        var nativeArray = UnsafeUtility.As<NativeArray<Matrix4x4>, NativeArray<float4x4>>(ref _array);
        _jobhandle = new UpdateDataJob((float) (DateTime.Now - _then).TotalMilliseconds)
        {
            array = nativeArray
        }.ScheduleBatch(totalObjects, 32);
    }

    [BurstCompile]
    struct UpdateDataJob : IJobParallelForBatch
    {
        [NoAlias] [NativeDisableParallelForRestriction]
        public NativeArray<float4x4> array;

        public UpdateDataJob(float time) : this() { _time = time * 0.001f; }

        public void Execute(int startIndex, int count)
        {
            for (var index = startIndex + count - 1; index >= startIndex; index--)
            {
                float j = index / numberOfObjectsPerSide;
                float i = index % numberOfObjectsPerSide;

                var y = math.sin(2 * math.PI * (i - _time) / 10);

                array[index] = float4x4.TRS(new float3(i * 1.5f, y, j * 1.5f), quaternion.identity
                                          , new float3(1, 1, 1));
            }
        }

        readonly float _time;
    }

    DateTime                    _then;
    NativeArray<Matrix4x4>      _array;
    GPUInstancerPrefabManager   _manager;
    GPUInstancerPrefabPrototype _prefabPrototype;
    ComputeBuffer               _transformationMatrixVisibilityBuffer;
    JobHandle                   _jobhandle;
}

This code won’t run unless you buy the GPUInstancer plugin. As I said GPUInstancer implements most of the rendering pipeline using Compute Shaders, however it still relies on Unity pipeline (in this case URP) for few things, like computing the lighting. Materials and shaders are also the ones you would use normally with Unity, with some slight variations to support Compute Buffer structured buffers. I won’t go into details about it, if you are interested just check the extensive GPUInstancer wiki page. Just know that it’s very simple to use even for a person like me who knows almost zero about compute shaders.

The #define OLD_WAY would be how you normally use the plugin. However, because how slow ComputeBuffer SetData is, uploading 1 million matrices would make this solution unpractical. I have already suggested to the GPUInstancer authors if they can put more thoughts about the data optimizations. a 4×4 matrix may be much more than I need to upload to the GPU (for example if I just need to translate the object).

The Job uses burst to transform 1 Million cubes, Burst will do a good job (no pun intended), considering that maybe that simple math could be even optimized:

however the ComputeBuffer.SetData method would take too long to upload the entire array of matrices

But what happens if I switch to a SubUpdates buffer? Well I have no clue what’s happening and no interest for now to learn it. This was just an exercise after all. I believe that, somehow, the Nvidia Driver let me directly write in to the memory that is used by the GPU to reserve the compute buffer. What I know is that I get this in the profiler:

which means plenty of CPU avaiable to even make a game!

In order to achieve this, I had to hack a bit the GPUInstancer code (which ships with the plugin). I just changed the buffer initialization to:

runtimeData.transformationMatrixVisibilityBuffer =
                    new ComputeBuffer(runtimeData.bufferSize, GPUInstancerConstants.STRIDE_SIZE_MATRIX4X4, ComputeBufferType.Structured, ComputeBufferMode.SubUpdates);

Careful though, I spent almost one hour to understand why Begin/EndWrite wasn’t working at first. Turned out that currently it works only with the Vulkan Renderer, although Unity documentation doesn’t mention this anywhere!

Edit: it seems that is not necessary to call Begin/EndWrite every frame, but only when needed. I am not sure about it honestly, so if someone could shed some light, it would be appreciated.