How to define a gltf format for meshlets?

[fire]

Is there any research in defining a meshlet format for gltf?

I'm thinking a gltf scene where each chunk of 126 triangles is a mesh.

[zeux]

Going to convert to a discussion, as there isn't any associated code changes that are actionable here.

I haven't looked into glTF storage for meshlet data specifically. I think Bevy folks might have at some point. But my general observation is that the meshlet data layout seems to be rather specific to a given rendering pipeline / engine, and I don't know to what extent there's a lot of unification possible.

Baseline meshopt algorithm expectation is that a meshlet is identified by a subset of 8-bit local indices and an indirection table that remaps vertex indices from local meshlet space to global vertex buffer. However, applications are free to process this data arbitrarily for actual storage, for example:

• The indices may be packed more tightly than 8-bit, or less tightly than 3 bytes per triangle (to ensure 4b aligned loads)
• The indirection table may be compressed in a variety of ways on a per meshlet basis
• The indirection table may be removed entirely at the cost of duplicating all boundary vertices (more memory used, but potentially shorter access latency)

Many renderers would want some type of extra meshlet data for culling: could be just a bounding volume (e.g. sphere), could be an additional cone for normal culling, all of this might be bitpacked.

Some renderers may decide to quantize the vertex data in meshlet local space, which further breaks the "vertex buffer" model as there's no concept of "vertex data" in isolation from a meshlet that embeds it.

And of course some renderers may want to use hierarchical cluster storage which requires even more data per meshlet and gets even more nuanced.

[fire]

If it helps I'm trying to investigate Godot Engine level streaming vs Godot Engine mesh streaming vs running a machine learning model that goes text-to-gltf in the style of vision transformers which cut the image into small grid cells like meshlets. Like generate vision tokens per meshlet.

These are three completely different problems with different tradeoffs.