Generate vocals on top of an existing instrumental, or an instrumental on top of existing vocals?

[iGerman00]

Some commercial music generation services include an option to basically replicate an input audio, then add vocals on top of it.
Would something like that be possible with YuE? This would allow, for example, producers to brainstorm and add vocals to their own music quickly.

Another thing is it would be nice to be able to have a reference for the tone of the vocals, to be able to save specific 'vocalists' in our workflows

[a43992899]

Yes it is possible without training. We will add it to todos.

[frink]

What about the opposite. You want to add instruments to an existing recording? That would be just as ground breaking!

[migperfer]

Hi! I'm working on this, but I think that I'm missing some details from the code alone.

The ids vocal and instrumental tokens, can they be the same? Or are they separated?
One example of what I mean:

• The total number of tokens is 400
• Vocal tokens ids are below 200
• Instrument tokens are above 200

[betweentwomidnights]

me trying to do this by encoding the instrumental and injecting it as the instrumental codebook, then generating vocals step-by-step...

https://github.com/betweentwomidnights/YuE-instrumental-injection

vibe coded as f

so it's a really slop implementation. I have however gotten it to generate vocals...it seems to mostly only want to rap in the current state though. Sometimes it'll produce some really insane outputs too.

I can tell I still haven't figured out the segmentation/context window properly while doing my workaround instead of model.generate()