Skip to content

Commit 89fce36

Browse files
docs(splats): keep package readme concise (#2968)
1 parent e6413e1 commit 89fce36

1 file changed

Lines changed: 10 additions & 194 deletions

File tree

modules/splats/README.md

Lines changed: 10 additions & 194 deletions
Original file line numberDiff line numberDiff line change
@@ -1,200 +1,16 @@
11
# @luma.gl/splats
22

3-
Experimental Gaussian splat rendering utilities for luma.gl. This package owns prepared GPU splat
4-
data, anisotropic Gaussian rendering, and backend-specific render models without depending on
5-
Apache Arrow, loaders.gl, or deck.gl.
3+
Experimental Gaussian splat rendering utilities for luma.gl, without dependencies on Apache Arrow,
4+
loaders.gl, or deck.gl.
65

7-
The package is a private luma.gl workspace and is not published to npm. Install dependencies from
8-
the repository root and reference it from another workspace with `"@luma.gl/splats": "workspace:*"`.
6+
`makeGPUSplatData(...)` prepares caller-owned GPU data. `SplatRenderer` supports WebGPU and WebGL2;
7+
`GPUSplatGraphRenderer` progressively streams preserved batches through reusable WebGPU command
8+
graphs, global GPU sorting, and one indirect draw.
99

10-
## Usage
10+
Use optional `expectedSplatCount` and `expectedBatchCount` hints to reserve graph capacity.
11+
Renderers borrow source batches, preserve HDR colors, and must be destroyed before their data.
1112

12-
```ts
13-
import {makeGPUSplatData, SplatRenderer} from '@luma.gl/splats';
13+
This private package is not published to npm. Reference it from another luma.gl workspace with
14+
`"@luma.gl/splats": "workspace:*"`.
1415

15-
const splatData = makeGPUSplatData(device, {
16-
positions: new Float32Array([0, 0, -2]),
17-
scales: new Float32Array([0.25, 0.12, 0.08]),
18-
rotations: new Float32Array([1, 0, 0, 0]),
19-
colors: new Uint8Array([235, 150, 80, 255]),
20-
opacities: new Float32Array([0.8])
21-
});
22-
23-
const renderer = new SplatRenderer(device, {data: splatData});
24-
25-
renderer.draw(renderPass);
26-
renderer.appendData(nextSplatBatch);
27-
28-
// Destroy borrowing renderers before destroying caller-owned prepared data.
29-
renderer.destroy();
30-
splatData.destroy();
31-
nextSplatBatch.destroy();
32-
```
33-
34-
`GPUSplatData` is caller-owned prepared data. Renderers borrow source buffers and release only their
35-
own models and temporary resources. Streaming retains source batch boundaries instead of
36-
concatenating or reuploading previously prepared batches.
37-
38-
Source colors can use normalized `Uint8Array` RGBA or linear `Float32Array` RGBA. Floating-point
39-
colors preserve high-dynamic-range radiance without premature clamping or quantization. The
40-
renderer automatically applies Reinhard highlight compression on standard dynamic range targets,
41-
preserves extended WebGPU presentation output, and supports explicit `exposure` and `toneMapping`
42-
controls.
43-
44-
## WebGPU command-graph rendering
45-
46-
`GPUSplatGraphRenderer` is a WebGPU-only alternative for large captured scenes. Its compiled GPU
47-
command graph projects and culls each borrowed source batch, globally sorts camera-dependent depth
48-
keys, and renders every visible Gaussian with one GPU-driven indirect draw:
49-
50-
```ts
51-
import {GPUSplatGraphRenderer, type GPUSplatData} from '@luma.gl/splats';
52-
53-
const renderer = new GPUSplatGraphRenderer(webgpuDevice, {
54-
viewportSize: [width, height],
55-
expectedSplatCount: 741_883,
56-
expectedBatchCount: 12
57-
});
58-
59-
const preparedBatches: GPUSplatData[] = [];
60-
for await (const batch of preparedBatchStream) {
61-
preparedBatches.push(batch);
62-
renderer.appendData(batch);
63-
64-
const commandEncoder = webgpuDevice.createCommandEncoder();
65-
renderer.encode(commandEncoder);
66-
webgpuDevice.submit(commandEncoder.finish());
67-
}
68-
69-
// Source batches belong to their caller and must outlive the borrowing renderer.
70-
renderer.destroy();
71-
for (const batch of preparedBatches) {
72-
batch.destroy();
73-
}
74-
```
75-
76-
The example reserves room for the complete 741,883-splat Train scene and its twelve Arrow batches.
77-
The first nonempty batch compiles the graph and renders immediately; later batches replace their
78-
reserved source bindings without replacing the graph. The resulting pipeline is:
79-
80-
```text
81-
Caller-owned, independently allocated batches One compiled GPU command graph
82-
83-
batch 0: five original GPU columns --------+
84-
batch 1: five original GPU columns --------+--> reusable imported source slots
85-
batch N: five original GPU columns --------+ |
86-
v
87-
initialize every reserved row ---------------> project and cull populated slots
88-
identity indices + invalid depth keys per-slot uniforms + 48-byte records
89-
reset indirect visible count |
90-
+--> global 16-bit radix sort --+
91-
| |
92-
+--> atomic visible count -----+
93-
|
94-
v
95-
one indirect draw
96-
```
97-
98-
### Source batches and reusable graph slots
99-
100-
Parsing remains outside this package: `@loaders.gl/splats` recognizes file formats, `@luma.gl/arrow`
101-
converts individual Apache Arrow record batches to `GPUSplatData`, and this renderer consumes the
102-
resulting framework-neutral GPU objects. Source batches stay separate and retain their original
103-
buffer allocations throughout loading, rendering, capacity growth, and destruction.
104-
105-
Each reserved graph slot represents exactly one source batch and imports five columns: positions,
106-
scales, rotations, colors, and opacities. Graph topology is immutable after compilation, so every
107-
slot initially has five distinct, renderer-owned placeholder buffers. Their minimum sizes are 12,
108-
12, 16, 4, and 4 bytes respectively. At encoding time, populated slots override those imported
109-
buffers with the original caller-owned allocations; unpopulated slots keep their harmless
110-
placeholders and do not dispatch projection work. This changes bindings, not graph topology, and
111-
never concatenates, repacks, copies, or reuploads existing source data.
112-
113-
A separate 128-byte uniform buffer per slot carries the camera transform, viewport, Gaussian
114-
styling, batch offset, active row count, color format, exposure, and tone-mapping mode. Updating
115-
these uniforms costs work proportional to the reserved batch-slot count, not the splat-row count.
116-
117-
### GPU execution phases
118-
119-
Each necessary `encode(commandEncoder)` records the following dependent graph operations into the
120-
caller's command encoder. The caller still owns submission; no intermediate submissions or
121-
synchronous GPU readbacks are required.
122-
123-
1. **Initialize reserved rows.** A compute pass writes stable identity indices across the entire
124-
reserved row capacity, initializes every depth key to the invalid sentinel `65,535`, and resets
125-
the indirect draw's visible-instance count to zero. Unfilled capacity therefore cannot be drawn.
126-
127-
2. **Project and cull active batches.** One compute node per reserved slot processes its source
128-
only when that slot contains a nonempty batch. Visible anisotropic Gaussians produce 48-byte
129-
camera-dependent records containing clip position, screen-space axes, and linear RGBA radiance.
130-
Invalid, clipped, transparent, or otherwise culled rows keep their sentinel keys. Valid rows
131-
receive far-to-near keys in the range `0` through `65,534`, and each increments the indirect
132-
visible-instance count atomically. Packed `Uint8Array` colors and floating-point HDR colors can
133-
coexist in different batches.
134-
135-
3. **Globally sort depth keys.** `GPUSort` performs a stable ascending radix sort using only the 16
136-
meaningful depth-key bits. Its values are global projected-record indices, so transparent
137-
Gaussians from different source batches share one correct back-to-front order. The maximum
138-
sentinel sorts culled and inactive rows after every visible record.
139-
140-
4. **Render once.** One graph-native indirect render pass consumes sorted projected-record indices
141-
and the GPU-written visible count. The fragment shader expands and blends anisotropic Gaussian
142-
quads, preserving source HDR radiance until exposure and display-appropriate tone mapping are
143-
applied. No source-batch rebinding or one-draw-per-batch ordering is required in this pass.
144-
145-
### Progressive capacity and graph lifetime
146-
147-
`expectedSplatCount` and `expectedBatchCount` are optional, positive-integer capacity hints. When
148-
the complete source metadata is known, supplying both reserves the final row and batch-slot
149-
capacities up front. The first nonempty `encode()` compiles one graph; appending any batch that
150-
fits both capacities reuses the same `renderer.compiledGraph` object and only updates its imported
151-
source bindings and uniforms.
152-
153-
Unknown or underestimated dimensions remain valid:
154-
155-
- Without a row hint, the initial capacity is at least four times the first nonempty batch size;
156-
without a batch hint, at least four source slots are reserved.
157-
- If incoming rows or batch slots exceed their current capacity, the corresponding capacity
158-
doubles until it can accommodate the accumulated stream. A new graph is compiled only at that
159-
growth boundary; subsequent fitting batches reuse it.
160-
- `renderer.capacity` exposes the current allocated row and slot counts, while `renderer.stats`
161-
reports actual loaded rows, actual loaded batches, and source-versus-renderer GPU memory.
162-
- Appending a batch or changing camera/style properties marks the graph dirty. Once the stream and
163-
camera are stationary, another `encode()` returns `undefined` instead of repeating projection,
164-
sorting, and drawing.
165-
166-
The lifecycle is therefore: no source rows means no graph; the first nonempty batch compiles and
167-
renders immediately; fitting batches stream through the same graph; capacity overflow occasionally
168-
recompiles; and a stationary completed scene performs no repeated GPU work. Replacing the entire
169-
source list with `setProps({data})` invalidates the old graph without destroying either the old or
170-
new caller-owned batches.
171-
172-
### Ownership, precision, and performance tradeoffs
173-
174-
`GPUSplatData` and all original source-column allocations remain caller-owned and must outlive the
175-
renderer. The renderer owns only derived projected records, depth keys, source indices, sorted
176-
keys/indices, radix-sort scratch, uniform buffers, slot placeholders, the indirect command, and its
177-
graph/model resources. Call `renderer.destroy()` before destroying the prepared source batches.
178-
179-
The baseline persistent working buffers require 64 bytes per reserved row: 48 bytes for each
180-
projected record and four 4-byte key/index arrays. Reserving the complete 741,883-splat Train scene
181-
therefore uses approximately 45.28 MiB before graph scratch, per-slot uniforms/placeholders, and
182-
the separately owned source columns. `renderer.stats` and `renderer.graphStats` expose the actual
183-
allocation and scheduling diagnostics without reading back source rows or visible counts.
184-
185-
The fixed-topology radix sort currently processes the **entire reserved row capacity on every dirty
186-
frame**, even while only the first streamed batch is populated. Exact final-size hints eliminate
187-
recompilation and improve CPU responsiveness, but front-load GPU memory and full-scene sorting
188-
work; unknown-size geometric growth trades occasional recompilation for less initially reserved
189-
capacity. Sixteen-bit keys reduce radix work but provide finite depth-ordering precision; the sort
190-
is stable when rows quantize to the same key.
191-
192-
Projected records also must fit the adapter's `maxStorageBufferBindingSize`. For the common 128 MiB
193-
binding limit, the 48-byte stride permits at most 2,796,202 projected rows in one graph; actual
194-
available GPU memory or other implementation limits may lower that practical ceiling. Applications
195-
should retain their existing fallback policy when an adapter cannot allocate the requested scene.
196-
197-
`GPUSplatGraphRenderer` requires WebGPU. Continue to use `SplatRenderer` for WebGL2 devices or for
198-
explicit CPU-ordered comparison; both renderers preserve caller ownership and original source-batch
199-
boundaries. Keep deck.gl-specific layers in downstream adapters instead of introducing framework
200-
or file-format dependencies into `@luma.gl/splats`.
16+
See [luma.gl](https://luma.gl/docs/api-reference/splats) for documentation.

0 commit comments

Comments
 (0)