3D Gaussian Splatting Explained w/gsplat's Ruilong Li

This episode of Computer Vision Decoded features a discussion on 3D Gaussian Splatting, a new paradigm in computer graphics. The hosts welcome Ruilong Li, a researcher at NVIDIA and former UC Berkeley student, who provides an expert explanation of Gaussian Splatting, including its components, how it works, and how to achieve good images. Ruilong discusses his background, transitioning from 2D to 3D computer vision, and the evolution from Nerfs to Gaussian Splatting. The discussion covers the basics of Gaussian Splatting, comparing it to mesh representations, and detailing how these 3D scenes are created from images through optimization and rendering. They also discuss the importance of image quality, camera poses, and various parameters that can be tuned to optimize the final output, as well as the differences between Nerfstudio and gsplat.

Outlines

Part 1: Introduction and Background

Part 2: Gaussian Splatting Explained

Part 3: Optimization and Refinement

Part 4: Alternative Approaches and Representations

Part 5: Nerfstudio, gsplat, and Community

Sign in to continue reading, translating and more.

Open full episode in Podwise

EveryPoint

Part 1: Introduction and Background

Introduction to Radiance Fields and Gaussian Splatting

Ruilong Li's Background and Journey into 3D Computer Vision

The Rapid Evolution of 3D Computer Vision and Introduction to Gaussian Splatting

Part 2: Gaussian Splatting Explained

Defining Gaussian Splatting: Gaussians and Splatting

Comparing Gaussian Splats to Mesh Representations

Details and Optimization of Gaussian Blobs

Geometry, Texture, and Input Requirements for Gaussian Splatting

Tools and Techniques for Camera Pose Estimation and Point Cloud Initialization

Dense vs. Sparse Point Clouds and Imagery Tips

Lighting Considerations and Transition to the Optimization Stage

Part 3: Optimization and Refinement

The Optimization Loop: Analysis by Synthesis

Iterative Refinement and Hyperparameter Tuning

Optimizing for Speed and the Evolution of Gaussian Splatting

Part 4: Alternative Approaches and Representations

Diminishing Returns and Alternative Primitives

Comparing Gaussians to Other Primitives and the Betamax Moment

The Dominance of Mesh Representations and the Purpose of 3D Representations

Combining Different Representations and the Conditional Generation Problem

Part 5: Nerfstudio, gsplat, and Community

Nerfstudio and gsplat: Origins, Differences, and Future Directions

Choosing Between Nerfstudio and gsplat and Permissive Licensing

Community Contributions and Modular Design

Windows Support, Future Episodes, and Closing Remarks

3D Gaussian Splatting Explained w/gsplat's Ruilong Li

EveryPoint

Part 1: Introduction and Background

00:00Introduction to Radiance Fields and Gaussian Splatting

Introduction to Radiance Fields and Gaussian Splatting

02:08Ruilong Li's Background and Journey into 3D Computer Vision

Ruilong Li's Background and Journey into 3D Computer Vision

06:01The Rapid Evolution of 3D Computer Vision and Introduction to Gaussian Splatting

The Rapid Evolution of 3D Computer Vision and Introduction to Gaussian Splatting

Part 2: Gaussian Splatting Explained

09:35Defining Gaussian Splatting: Gaussians and Splatting

Defining Gaussian Splatting: Gaussians and Splatting

12:12Comparing Gaussian Splats to Mesh Representations

Comparing Gaussian Splats to Mesh Representations

15:03Details and Optimization of Gaussian Blobs

Details and Optimization of Gaussian Blobs

18:06Geometry, Texture, and Input Requirements for Gaussian Splatting

Geometry, Texture, and Input Requirements for Gaussian Splatting

21:16Tools and Techniques for Camera Pose Estimation and Point Cloud Initialization

Tools and Techniques for Camera Pose Estimation and Point Cloud Initialization

24:55Dense vs. Sparse Point Clouds and Imagery Tips

Dense vs. Sparse Point Clouds and Imagery Tips

27:44Lighting Considerations and Transition to the Optimization Stage

Lighting Considerations and Transition to the Optimization Stage

Part 3: Optimization and Refinement

30:28The Optimization Loop: Analysis by Synthesis

The Optimization Loop: Analysis by Synthesis

34:38Iterative Refinement and Hyperparameter Tuning

Iterative Refinement and Hyperparameter Tuning

37:57Optimizing for Speed and the Evolution of Gaussian Splatting

Optimizing for Speed and the Evolution of Gaussian Splatting

Part 4: Alternative Approaches and Representations

40:41Diminishing Returns and Alternative Primitives

Diminishing Returns and Alternative Primitives

43:39Comparing Gaussians to Other Primitives and the Betamax Moment

Comparing Gaussians to Other Primitives and the Betamax Moment

48:42The Dominance of Mesh Representations and the Purpose of 3D Representations

The Dominance of Mesh Representations and the Purpose of 3D Representations

53:00Combining Different Representations and the Conditional Generation Problem

Combining Different Representations and the Conditional Generation Problem

Part 5: Nerfstudio, gsplat, and Community

58:29Nerfstudio and gsplat: Origins, Differences, and Future Directions

Nerfstudio and gsplat: Origins, Differences, and Future Directions

1:03:40Choosing Between Nerfstudio and gsplat and Permissive Licensing

Choosing Between Nerfstudio and gsplat and Permissive Licensing

1:07:05Community Contributions and Modular Design

Community Contributions and Modular Design

1:12:40Windows Support, Future Episodes, and Closing Remarks

Windows Support, Future Episodes, and Closing Remarks