Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please make high resolution time available within AudioWorkletGlobalScope. #2413

Open
issuefiler opened this issue May 13, 2020 · 31 comments · May be fixed by w3c/hr-time#169
Open

Please make high resolution time available within AudioWorkletGlobalScope. #2413

issuefiler opened this issue May 13, 2020 · 31 comments · May be fixed by w3c/hr-time#169
Labels
category: new feature Substantive changes that add new functionality. https://www.w3.org/policies/process/#class-4 Needs External Dependency Resolution An external dependency is blocking progress on this issue. Priority: Soon Significant interest; "want to have". https://speced.github.io/spec-maintenance/about/ size: S Small amount of work expected to resolve.

Comments

@issuefiler
Copy link

The desired feature

High resolution time available within AudioWorkletGlobalScope.

This specification defines an API that provides the time origin, and current time in sub-millisecond resolution, such that it is not subject to system clock skew or adjustments.


I’d love to see (self.performance.now() + self.performance.timeOrigin) instead of Date.now() in my processor code, as my process function is called every other third milliseconds at the sample rate of 192000 Hz.


Note

Currently there’s no way for my processor to use it.

DOMException: Failed to execute 'postMessage' on 'MessagePort': Performance object could not be cloned.
@kawogi
Copy link

kawogi commented May 13, 2020

Additionally: new Date() is an allocation (unless optimized out) which should not be done in a Worklet.
Misread the code, sorry.

@padenot
Copy link
Member

padenot commented May 14, 2020

This makes sense. We need to look at the performance object to see if we want everything in it (probably not).

@guest271314

This comment was marked as off-topic.

@issuefiler
Copy link
Author

@guest271314

……dedicated MessagePort can be created specifically to post result of performance.now() to AudioWorkletGlobalScope

We could do that, but messaging with each other takes extra time. It is the process function in an AudioWorkletGlobalScope that knows the most accurate timing to stamp the time on samples.

@guest271314

This comment was marked as off-topic.

@issuefiler
Copy link
Author

Right. For my personal use case, I need to know (more) exactly what time the processed samples correspond to, and it turned out just performance.now() available would not be enough for that as the samples are buffered to be processed. We could look for something that’ll provide detailed and reliable information about the timings and more.

Tracing AudioWorkletHandler::Process
Worker feeding timestamps

@guest271314

This comment was marked as off-topic.

@guest271314

This comment was marked as off-topic.

@padenot
Copy link
Member

padenot commented Jun 16, 2020

Virtual F2F:

@padenot
Copy link
Member

padenot commented Oct 19, 2020

TPAC 2020:

  • Precise timers are still problematic because they allow side channels still. It might be possible to hide this behind COOP/COEP.
  • Real performance profiling tools (real-time safe) are now available and are bound to be equal or better than what can be done by authors. This was not the case when the comment above was made.
  • A view of load from the control thread is still desirable (Add detail of connecting audio node to non audio node #40), because having performance.now() doesn't allow measuring native nodes.

@padenot
Copy link
Member

padenot commented Nov 30, 2020

Precise timers are still problematic because they allow side channels still. It might be possible to hide this behind COOP/COEP.

This is what w3c/hr-time#89 is about, it seems possible. hr-time should offer a hook that says "if this document is isolated, timers are precise".

@brutzman
Copy link

brutzman commented Mar 11, 2021

If repeatably feasible, use of high-resolution time values will likely be valuable for X3D Graphics use of Web Audio API for presentation of acoustics. (This is speculative, potential future use case.)

@rtoy
Copy link
Member

rtoy commented Mar 11, 2021

Teleconf: It seems the primary use case (not including https://github.com/WebAudio/web-audio-api-v2/issues/77#issuecomment-796910897), is for performance evaluation. Perhaps this is covered in WebAudio/web-audio-api-v2#40, in which case there may not be a need for this in a worklet.

Leave this open, but reduce priority, pending more use cases that aren't related to performance measurements.

@rtoy
Copy link
Member

rtoy commented Apr 30, 2021

@padenot
Copy link
Member

padenot commented May 21, 2021

AudioWG virtual F2F:

  • This is being requested by lots of people, we need to do it

@mdjp mdjp transferred this issue from WebAudio/web-audio-api-v2 Sep 29, 2021
@mdjp mdjp added the category: new feature Substantive changes that add new functionality. https://www.w3.org/policies/process/#class-4 label Sep 29, 2021
@hoch
Copy link
Member

hoch commented Nov 8, 2022

@juj also asked this in #2527. I think we should get the spec change first since the actual spec work might be very small.

@Night12138
Copy link

Hi everyone, how is the progress going? We need to know the current render capacity, and a high-precision clock is required.

@hoch
Copy link
Member

hoch commented Sep 13, 2023

2023 TPAC Audio WG Discussion:
Because this involves the high resolution timer on a high priority thread, each implementer will first have an internal security discussion. The discussion will continue once the implementers are confident in changing the spec and beginning implementation.

@juj
Copy link

juj commented Sep 13, 2023

I thought the process isolation mechanism addressed high precision timer security issues?

In any case, it is possible today to create a high precision timer by creating an AudioWorklet with a SharedArrayBuffer, in an AudioContext with sampling rate of 192000, and in the process function, just ++ an integer in the SAB. That gives you a somewhat precise 0.666msec granular timer, and that can be used to polyfill performance.now() in absence of the real thing. Exposing performance.now() today would be weaker than that.

The need to get performance.now() into AudioWorklets at least for the Emscripten community is of ergonomy, so that we don't need to create polyfills in different scopes so that wasm compiled code can run in all threads the same.

@guest271314

This comment was marked as off-topic.

@padenot
Copy link
Member

padenot commented Sep 20, 2023

Because this involves the high resolution timer on a high priority thread, each implementer will first have an internal security discussion. The discussion will continue once the implementers are confident in changing the spec and beginning implementation.

With my Firefox implementer hat on, and after talking internally:

In Firefox, iff if the page is able to instantiate SharedArrayBuffer (e.g. because the headers have been set when loading the document), then the precision of performance.now() is 20us, which might well be good enough for our purpose here.

@hoch
Copy link
Member

hoch commented Feb 29, 2024

@padenot What would be the next step to make this possible? It looks like a great addition that is reasonably popular.

@sbc100
Copy link

sbc100 commented Nov 20, 2024

This issue came up again recently on the emscripten side: emscripten-core/emscripten#22962.

@mjwilson-google
Copy link
Contributor

We now have a Chromium issue that is asking for this and also said that Microsoft Edge has already gone ahead and implemented the Performance API in worklets.

It seems like there is support for this overall so we should probably spec it out soon. I'll add it to the agenda for next WG meeting.

@mjwilson-google mjwilson-google added Needs Discussion The issue needs more discussion before it can be fixed. Priority: Soon Significant interest; "want to have". https://speced.github.io/spec-maintenance/about/ size: S Small amount of work expected to resolve. Agenda+ Should be discussed at the next WG meeting. https://speced.github.io/spec-maintenance/about/ labels Jan 28, 2025
@padenot
Copy link
Member

padenot commented Feb 4, 2025

To follow-up to what @mjwilson-google says, this would need to happen here: https://w3c.github.io/hr-time/#sec-performance

Just a PR that adds , AudioWorklet, after in the Exposed section.

@chrisguttandin
Copy link
Contributor

I tried to check if this is really implemeted in Edge. I ran my test in version 132.0.2957.140 on macOS. This is the code that I used.

class TestProcessor extends AudioWorkletProcessor {
    constructor () {
        super();

        console.log(typeof performance);
    }

    process() {
        console.log(typeof performance);

        return true;
    }
}

console.log(typeof performance);

registerProcessor('test-processor', TestProcessor);

It logged 'undefined' all the time. Maybe I'm testing for the wrong thing but it looks like the Performance API isn't available in Edge either.

@padenot
Copy link
Member

padenot commented Feb 5, 2025

@chrisguttandin did you try to run this in isolated mode as well?

@mjwilson-google
Copy link
Contributor

Even if it's not actually supported in Edge yet we should still do this. Maybe I misunderstood the issue report.

this would need to happen here: https://w3c.github.io/hr-time/#sec-performance

So it's actually outside our spec? I can raise the PR if we're all in agreement.

@padenot
Copy link
Member

padenot commented Feb 5, 2025

So it's actually outside our spec? I can raise the PR if we're all in agreement.

Yes, this spec I linked defines where their stuff is exposed, but we're the one that can ask for it to be exposed, if that makes sense.

@chrisguttandin
Copy link
Contributor

@chrisguttandin did you try to run this in isolated mode as well?

@padenot I set the COEP/COOP header pair and also made sure that SharedArrayBuffer is available.

@mjwilson-google
Copy link
Contributor

we're the one that can ask for it to be exposed, if that makes sense.

Makes sense to me! I raised w3c/hr-time#169 -- please add supporting comments there if necessary.

@mjwilson-google mjwilson-google added Needs External Dependency Resolution An external dependency is blocking progress on this issue. and removed Needs Discussion The issue needs more discussion before it can be fixed. Agenda+ Should be discussed at the next WG meeting. https://speced.github.io/spec-maintenance/about/ labels Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: new feature Substantive changes that add new functionality. https://www.w3.org/policies/process/#class-4 Needs External Dependency Resolution An external dependency is blocking progress on this issue. Priority: Soon Significant interest; "want to have". https://speced.github.io/spec-maintenance/about/ size: S Small amount of work expected to resolve.
Projects
None yet
Development

Successfully merging a pull request may close this issue.