HuggingFace Repo -- Guidance for how to transform tokens to audio tensors when streaming

[HarrisonUnifyAI]

I have switched over to using the Hugging Face compatible model. Thank you guys for that. Excellent work!

I am trying to stream the generation to improve latency in a demo I have written using the Optional["BaseStreamer"] interface. Integration with that has gone smoothly but it streams the tokens, not the audio tensors, so when I naively tried to play the streamed data it was of course not working.

Is there a recommended way to process these tokens or some documentation somewhere I may have missed?

I have seen the csm-streaming repo, but not sure I can translate that code (which uses this python repo) to work with the new hugging face/transformers

[ZackHodari]

All the HF implementation was done by them, it's a great contribution. If you want more guidance you'll have to ask on their forums

As for streaming, conceptually you'll need the codec to work in a streaming fashion too. Mimi supports this, but I'm not sure what HF uses for the codec decoding

[jeffDebug]

@HarrisonUnifyAI Hi, have you asked the HF guys and get any answers about this question?