Tearing Artifacts in Rendered Videos During "Eval with 3DGS Rendering"

[ChunxunTian]

Thanks for your excellent work and for making the code publicly available.

I have reproduced the cloth and sloth categories using the provided data and checkpoints, following the README instructions. However, during the "Eval with 3DGS Rendering" stage, I observed tearing artifacts in the rendered videos. Specifically, the gripper appears to pull only a few local particles away from the object, causing it to look torn apart, rather than moving the entire object as a whole. An example has been attached to the issue. I would like to know why this issue occurs and how it can be resolved. If necessary, I can provide additional relevant information.

Besides, I noticed that the reconstructed 3D Gaussians provided in the downloaded datasets are not complete objects but only parts of the objects. Could this affect the rendering?

Thank you for your support!

Best Regards

episode_0610.mp4

[kywind]

Hi, thanks for trying out our code!

It appears the pipeline is functioning correctly. Regarding the tearing artifacts in the video, this behavior is caused by a bunch of factors, for example, (1) the errors of the dynamics model, and (2) the limited Gaussian quality (since all the gaussians are trained using 4 camera views), and (3) inaccuracies in recorded robot position (due to data noise).

You can also check the point-only visualizations under eval/.../.../pv/, which can be more intuitive. Overall I think this type of error is expected because cloth is one of the more challenging objects, and there are a lot of occlusion scenarios in the dataset, making the problem harder.

Also currently the Gaussian files in the dataset only covers the evaluation set.