What is Gaussian Splatting?
3D Gaussian Splatting is a cutting-edge technique for representing and rendering radiance fields in real-time, introduced in 2023 by Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. It replaces traditional mesh or voxel representations with a cloud of anisotropic 3D Gaussians — each acting as a tiny volumetric light emitter — enabling fast, photorealistic, and interactive scene reconstruction from multi-view imagery.
Technical Definition & Background
3D Gaussian Splatting is a real-time rasterization technique that represents scenes using a large number of anisotropic 3D Gaussians instead of polygons or voxels. Each Gaussian is treated as a small volumetric emitter with soft edges and view-dependent appearance.
- Rasterization-based: Like triangles in traditional 3D graphics, splats are projected and drawn, but as ellipsoidal blobs rather than flat surfaces.
- Differentiable: The rasterizer is optimized to be differentiable during training, allowing use of SGD for parameter learning.
- Real-Time: After training, millions of splats can be rendered at over 100 FPS in 1080p.
In training, a sparse point cloud from Structure-from-Motion (SfM) is converted to Gaussians, which are then optimized using stochastic gradient descent. Over time, the system automatically splits, clones, or prunes Gaussians to achieve optimal density and detail fidelity.
Comparison to Meshes & NeRFs
Here’s how 3D Gaussian Splatting stacks up against traditional 3D mesh rendering and Neural Radiance Fields (NeRFs):
| Feature | 3D Meshes | NeRFs | Gaussian Splatting |
|---|---|---|---|
| Representation | Polygons (triangles) | Continuous neural field (MLP) | Unstructured 3D Gaussians |
| View-Dependent Effects | Limited | Excellent | Excellent (Spherical Harmonics) |
| Training Time | N/A (manual or photogrammetry) | ~12h–48h | ~5–30 min |
| Rendering Speed | Real-time | 1–5 FPS (slow) | 30–135 FPS (real-time) |
| Output Size | Small (< 100 MB) | Large (GBs) | Medium (300MB–1GB) |
| Runtime Requirement | GPU friendly | Requires NN inference | Lightweight GPU rendering |
Why Gaussian Splatting is So Powerful
- Real-Time Rendering: Up to 135 FPS with state-of-the-art visual quality.
- Fast Training: As low as 6–10 minutes to reach high-quality output.
- No Neural Network Needed at Runtime: Compact representation with no MLP inference at render time.
- Highly Compact: Only 1–5 million Gaussians are needed for complex scenes.
- Differentiable & Editable: Perfect for integration with training pipelines or tools like Splat Editors.
The Gaussian Splatting Workflow
- Image Capture: Take multiple overlapping photos of a static scene or object.
- Camera Calibration (SfM): Use Structure-from-Motion to generate camera poses + a sparse point cloud.
- Initialization: Generate initial 3D Gaussians based on the sparse point cloud.
- Training: Optimize the position, color (SH), opacity, and shape (covariance) of each Gaussian.
- Rendering: Use a custom tile-based GPU rasterizer to blend Gaussians in real-time using alpha compositing.
Who Created It & When?
3D Gaussian Splatting was introduced in the 2023 SIGGRAPH paper titled:
“3D Gaussian Splatting for Real-Time Radiance Field Rendering” by:
- Bernhard Kerbl (Inria, Université Côte d’Azur)
- Georgios Kopanas (Inria, Université Côte d’Azur)
- Thomas Leimkühler (Max-Planck-Institut für Informatik)
- George Drettakis (Inria, Université Côte d’Azur)
Is Gaussian Splatting AI?
It's not AI in the traditional neural network sense — there's no deep learning model at runtime. Instead, it uses optimization techniques (like gradient descent and backpropagation) to learn a scene representation.
Think of it as machine learning-powered rendering — fast, differentiable, efficient, and highly interactive.