Question about inference speed and VRAM used 推理速度及显存打印与实际不符问题

[LeeKeyu]

作者您好，非常感谢您分享这个工作！我尝试使用distilled模型推理，V2V模式，直接在40GB A800上跑报错OOM
然后加上了low vram参数
--enable_low_vram_mode
--group_offloading_type "leaf_level"
看到代码打印出来 Max memory: 5.709 GB，但实际查看nvitop显示的实时显存消耗是 22GB，请问这个差异是为什么呢？另外推理速度是1.8s/it，每段3步去噪则总共需要~6s/33帧，与论文中的19.5 FPS相去甚远。请问下，你们提到的19.5 FPS是指33帧除以3步总推理时间吗？（而不是latency的倒数）

[qqingzheng]

Please refer to: #47

[LeeKeyu]

Please refer to: #47

Hi, thank you for your reply! It seems the inference speed issue #3 is no longer available. From other issues, I find an inference time of 1-2s for 33 frames, but I got 5-6s on A800 for 3-step inference and wonder if this is normal.
To enable low vram, I added the params you suggested which should work for 6GB-only machine, while the actual VRAM used on my A800 GPU server was about 22GB according to nvitop and I am confused.
Thanks a lot for your help!

[qqingzheng]

1. 是的，19.5 FPS是指在单卡H100上，33 frames / 1-2s得到的结果。
2. 显存这个可能是diffusers后端的原因，具体可参考https://huggingface.co/docs/diffusers/main/en/optimization/memory#group-offloading