Gaussian Splatting is a relatively new rendering method that renders 3D images through a collection of Gaussians rather than meshes. This project is one that focuses on creating a method to render these splats in Unity’s Spatial.io platform, which has unique limitations due to its web-based platform.
For instance, Spatial.io doesn’t allow for the use of compute buffers to manage our data. This means that we are forced to use textures to load, sort, and ultimately render the data. Below is an example of what these textures look like when loaded with data from a splat .ply.
Through using this method of data management, it has become feasible to load and render a Gaussian splat in both Unity and Spatial.io despite the major restrictions. One issue that remains, however, is that Spatial.io has a different render pipeline that is not well-documented by its creators. When using their built-in Projection and View matrices to convert our 3D Gaussians to Clip space, there is still an error in the process, despite it working perfectly within Unity.
The reason for this issue is still being investigated, although the current belief is that there is an issue with Spatial.io’s View and Projection Matrices when ran through HLSL.
Kaelen Raible 