QEMU Extended Audio Support#1457
Conversation
|
For starters, one thing that I know will need to be changed here is how the audio is being played. I assume it's raw PCM data coming over the socket. And I am not sure if the EDIT: https://github.com/samirkumardas/pcm-player might be an interesting way to playback the data. It's an MIT license. I imagine the implementation could look something like this: _handleQEMUExtAudioMsg() {
if (this._sock.rQwait("QEMU extended audio message", 3, 1)) { return false; }
this._sock.rQshift8(); // for now there is only a single submessage type 1
const operation = this._sock.rQshift16();
if (operation === 1) { // stream is starting
this._player = new PCMPlayer(this._audio_opts); // set when RFB.enableAudio() is called
} else if (operation === 0) { // stream is stopping
this._player.destroy()
} else { // stream data
const length = this._sock.rQshift32();
const data = this._sock.rQshiftBytes(length);
this._player.feed(data);
}
return true;
} |
|
Hey all. For what it's worth, another OSS project called Neko managed to implement |
That actually looks interesting. I'm surprised I didn't come across it when I was working on my side project which is very similar. I'll poke through the code later, but on the README they mention that it uses WebRTC. If that is the case for the audio as well, I am not sure if it would be a good example for implementing here. WebRTC is definitely the easiest way to do something like this, but there are infrastructure requirements that I don't think would be available in most use cases. |
|
Thanks for the PR! Regarding a server for testing - from what I can gather it seems only the built in VNC server in libvirt/QEMU supports this. |
|
Yea that's what I gathered too from searching about. So in the context of the project where I am using noVNC, I was either going to try to figure out a way to "proxy" the protocol and inject audio messages...or just spin up some qemu VMs at some point and start tinkering. I spent like an hour trying to conceptualize the former and at the time just said "nah" 😛. |
|
Just chiming in that I haven't completely forgotten about this. I may try out some testing in the coming weeks. |
|
Good to hear @tinyzimmer |
|
@tinyzimmer: Any progress on getting qemu sound channel to work with noVNC? Anything I can do to help? |
|
@jgforbes I've been all over the place between starting new jobs and personal crises so I haven't had a chance to play with it extensively yet. If you want to jump it at any point by all means, but I promise I will try to get back to this eventually. Some considerations I've already encountered. The code in it's current state will not work I know that. Not without either refactoring the client to take raw PCM (lots of data), or implementing a server that can encode the audio to webm. If it's just the raw QEMU support we want to consider including anyway, then the code in this PR that calls the browser Audio APIs needs to be refactored accordingly, but the protocol stuff should be okay so far. Then it's just a matter of testing. Which basically using |
|
@tinyzimmer Would it be possible to contact you about your QEMU VNC audio channel work outside of github? |
|
Sure via slack or something like that? We could meet on IRC somewhere also if you'd like.
|
|
I made the slack channel #vm-viewers.I am not a slack user so I will have to see how this works.I assume that github removes email address from this type of messaging.Jeffrey Forbes
…-------- Original Message --------
Subject: Re: [novnc/noVNC] QEMU Extended Audio Support (#1457)
From: Avi Zimmerman <notifications@github.com>
Date: Wed, December 09, 2020 12:02 am
To: novnc/noVNC <noVNC@noreply.github.com>
Cc: jgforbes <jgf@twicinc.com>, Mention <mention@noreply.github.com>
Sure via slack or something like that? On Tue, Dec 8, 2020, 22:30 jeffforbes <notifications@github.com> wrote: > @tinyzimmer <https://github.com/tinyzimmer> Would it be possible to > contact you about your QEMU VNC audio channel work outside of github? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#1457 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AJFREM4SLXAEQV6JKPFZPZTST2EIFANCNFSM4QNDXCDQ> > . > —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@tinyzimmer Did you receive the email I sent to the address in your profile? |
|
@jeffforbes I must have missed it, sorry. That is the correct address though. In all fairness, I haven't had time, and not sure when or if I will, to continue work on this PR in the near future 😢 |
|
I'm tempted to close this PR. If anything because if I do approach it again I'd like to do so from a fresh slate. But also I am getting to the point in my project where I am adding support for QEMU machines on the server side. When I then wanted a dynamic display resolution, it made more sense to try to use Spice for those instances then to continue with noVNC. Though I think I'll try to keep support for both. If and when I do have that finished, I'll actually have a decent test environment for if I do want to play with this again. And, at least for me, it'll make more sense to pick it back up then. |
|
Understandable. Note however that noVNC does have support for dynamic display resolution using the ExtendedDesktopSize pseudo-encoding. I don't know if QEMU's VNC server has support for that though. |
|
I am actually using that already for the current functionality I have that doesn't use QEMU. But to my understanding, I don't think the QEMU VNC server supports it. I think that's where they push you towards SPICE instead. |
|
The QEMU project has put a lot of effort into the Spice protocol and the standalone spice viewer.Unfortunately, the HTML5 Spice viewer has not received any attention since it was first released anddoes not support many of the Spice features.Jeffrey Forbeshttp://www.linkedin.com/in/jeffreygforbes
…-------- Original Message --------
Subject: Re: [novnc/noVNC] QEMU Extended Audio Support (#1457)
From: Avi Zimmerman <notifications@github.com>
Date: Tue, February 16, 2021 1:57 am
To: novnc/noVNC <noVNC@noreply.github.com>
Cc: jgforbes <jgf@twicinc.com>, Mention <mention@noreply.github.com>
I'm tempted to close this PR. If anything because if I do approach it again I'd like to do so from a fresh slate. But also I am getting to the point in my project where I am adding support for QEMU machines on the server side. When I then wanted a dynamic display resolution, it made more sense to try to use Spice for those instances then to continue with noVNC. Though I think I'll try to keep support for both. If and when I do have that finished, I'll actually have a decent test environment for if I do want to play with this again. And, at least for me, it'll make more sense to pick it back up then. —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Yea I am noticing that as I try to integrate it. Will see where this journey takes me. |
|
It replaces: |
|
This PR: #1952 replaces: Linked to: |
6 participants
I opened a fresh draft because I had way too many mistakes on my first branch 😛
I have not yet tested these changes and obviously documentation will need to be updated. This is mainly intended to continue the discussion from #302.
Forgive me if I made some mistakes, since this is my first time really digging into the underlying protocol, but this is my first draft attempt at implementing handling for the QEMU Audio Server Message and public methods wrapping the QEMU Audio Client Message.
I'm trying to figure out the best way I could go about testing these changes and fine tuning them. In my current project where I'm using noVNC I am simply proxying the data from an Xvnc server, so I would need to find a good one to use that supports the extended protocol.