Mashaan blog

3D Gaussian Splatting

Acknowledgment:

I borrowed some code from gaussian-splatting-colab.

References:

@Article{kerbl3Dgaussians,
 author       = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
 title        = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
 journal      = {ACM Transactions on Graphics},
 number       = {4},
 volume       = {42},
 month        = {July},
 year         = {2023},
 url          = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
}

@article{ye2024gsplatopensourcelibrarygaussian,
 title    = {gsplat: An Open-Source Library for {Gaussian} Splatting},
 author   = {Vickie Ye and Ruilong Li and Justin Kerr and Matias Turkulainen and Brent Yi and Zhuoyang Pan and Otto Seiskari and Jianbo Ye and Jeffrey Hu and Matthew Tancik and Angjoo Kanazawa},
 year     = {2024},
 eprint   = {2409.06765},
 journal  = {arXiv preprint arXiv:2409.06765},
 archivePrefix= {arXiv},
 primaryClass = {cs.CV},
 url      = {https://arxiv.org/abs/2409.06765},
}

Gaussians in 1D, 2D, and 3D

1D Gaussian

1D-gaussian

2D Gaussian

2D-gaussian

3D Gaussian

3D-gaussian

Input to 3DGS algorithm

Given camera positions and sparsely scattered 3D points:

drawings-01 002

We initialize 3D gaussians centered at each one of these 3D points. The optimization process involves:

drawings-02 001

3DGS Forward and backward computation graphs

Forward and backward computation graphs as illustrated in Figure 4 in gsplat paper:

drawings-01 003

source: (Ye et al., 2024)

Forward and backward passes

Optimizing color

drawings-02 002

Optimizing opacity

drawings-02 003

Optimizing mean

drawings-02 004

Optimizing rotation and scale

drawings-02 005

Training 3DGS

%cd /content
!git clone --recursive https://github.com/camenduru/gaussian-splatting
!pip install -q plyfile

%cd /content/gaussian-splatting
!pip install -q /content/gaussian-splatting/submodules/diff-gaussian-rasterization
!pip install -q /content/gaussian-splatting/submodules/simple-knn

!wget https://huggingface.co/camenduru/gaussian-splatting/resolve/main/tandt_db.zip
!unzip tandt_db.zip

!python train.py -s /content/gaussian-splatting/tandt/truck

Viewing 3DGS output

After training, you can find a .ply file under /content/gaussian-splatting/output/{some numbers}/point_cloud/iteration_30000/point_cloud.ply

image

Go to: https://antimatter15.com/splat/ and drag the .ply file and drop it onto the viewer.

image