AUv2: request_resize reenters the plugin, ignores the calling thread, and always reports success

[defiantnerd]

Found while reviewing #572. That one was a VST3 problem — an inverted tresult and a cached rect handed to the host while a reentrant getSize was writing to it. AUv2 does not have that shape, but the same corner is wrong here in a different way, and AUv2 is the only one of the four wrappers left with it.

The call chain

plugin -> clap_host_gui.request_resize(w, h)
  WrapAsAUV2::gui_request_resize          src/detail/auv2/auv2_base_classes.h:678
  auv2shared_mm_request_resize            src/detail/auv2/auv2_shared.mm:10
  [nsv setFrame:NSMakeRect(0, 0, w, h)]   src/detail/auv2/auv2_shared.mm:15
  -setFrame: override                     src/detail/auv2/wrappedview.asinclude.mm:194
  gui->set_size(plugin, w, h)             src/detail/auv2/wrappedview.asinclude.mm:204

_window holds the CLAP_WRAPPER_COCOA_CLASS_NSVIEW instance, and that class overrides -setFrame:, so the message lands on the override and set_size() goes straight back into the plugin — from inside the plugin's own request_resize() call, with the numbers it just passed in.

1. The plugin is reentered synchronously

Every other wrapper deliberately avoids this:

• VST3 calls IPlugFrame::resizeView and lets the host come back through onSize later. That is the sequence Fix VST3 request_resize #572 just aligned with.
• AUv3 hands the request to the main queue (src/detail/auv3/auv3_audiounit.mm:577) and returns.
• The standalone forwards to onRequestResize and returns false when there is nothing wired up.

A plugin asks for 800x600 and gets set_size(800, 600) called on it before request_resize has returned. Editors that recompute their layout in set_size and ask again for the size they settle on will bounce between the two.

src/detail/auv2/wrappedview.asinclude.mm:29 already declares the flag for this:

bool setSizeByZoom;  // use this flag to see if resize comes from here or from external

It is never read or written anywhere in the tree. Same for underlyingUISize and lastScale on the two lines above it.

2. Nothing marshals to the main thread

request_resize is [thread-safe] in clap/ext/gui.h, so a plugin may call it from a worker. AUv2 then calls -setFrame: off that thread, which AppKit does not allow, and set_size() off it as well, which CLAP marks [main-thread].

Since #536 the -setFrame: override also takes the AU main-thread mutex, and the SDK holds that across its whole non-realtime dispatch, so an off-thread request_resize can park a plugin worker behind host initialization.

VST3 checks the thread id and defers through an atomic serviced in onIdle (src/wrapasvst3.cpp:1665). AUv3 dispatches. AUv2 does neither.

3. The answer to the plugin is always true

src/detail/auv2/auv2_shared.mm:17 returns true unconditionally and gui_request_resize passes it through, so the plugin never finds out the host did nothing with the request.

That also contradicts gui_can_resize() four lines above it (src/detail/auv2/auv2_base_classes.h:674), which returns false. The wrapper tells the plugin the host cannot resize, then tells it the resize worked.

4. setFrame: does not ask adjust_size

src/detail/auv2/wrappedview.asinclude.mm:204 pushes the host's size into set_size as-is. VST3 runs it through adjust_size in both onSize and checkSizeConstraint and uses what comes back, and #565 did the same for AUv3.

Suggested shape

• Marshal the request to the main thread. Either dispatch_async onto the main queue the way AUv3 does, or park it and service it from the idle tick the way VST3 does — AUv2 already has onIdle().
• Use setSizeByZoom for what it was declared for: set it around the wrapper-initiated -setFrame: so the override skips the set_size echo, and clear it after. A resize the host initiates still has to reach the plugin.
• Run the size through adjust_size in -setFrame: before calling set_size, and honour the result.
• Decide what the return value should say. AUv2 Cocoa gives no synchronous answer from the host, so either keep returning true — clap/ext/gui.h allows that for a request that is handled asynchronously, provided the host calls set_size to revert if it cannot satisfy it — or make gui_can_resize() honest first. Right now the two disagree with each other.

Note for whoever picks this up

ui_connection::_window is declared clap_window_t * but carries the NSView. That is the contract on AUv2, not an oversight — the cast in at wrappedview.asinclude.mm:99 and back out at auv2_shared.mm:14 is deliberate and the compiler cannot check it. Leave it alone.

Building and checking

macOS only, AUv2 is not built anywhere else.

cmake -S . -B build -GNinja -DCMAKE_BUILD_TYPE=Debug \
      -DCLAP_WRAPPER_DOWNLOAD_DEPENDENCIES=TRUE \
      -DCLAP_WRAPPER_BUILD_AUV2=TRUE \
      -DCLAP_WRAPPER_OUTPUT_NAME=testplug
cmake --build build --config Debug

and the clap-first component through auval, which is what CI runs:

cmake -S . -B build -DCLAP_WRAPPER_BUILD_TESTS=TRUE
cmake --build build --config Debug --target clap-first-distortion_all
mkdir -p ~/Library/Audio/Plug-Ins/Components/
cp -R "build/clap-first-distortion_assets/ClapFirst Bad Distortion.component" ~/Library/Audio/Plug-Ins/Components/
killall -9 AudioComponentRegistrar || true
auval -strict -v aufx BdDt FrAD

The wrapper targets build with -Wall -Wextra -Wpedantic -Werror, and commits have to be clang-formatted against .clang-format.

auval never opens the editor, so it will not exercise any of this. It is there to show nothing else broke. Confirming the fix itself needs a real host — Logic or Live with a plugin that resizes itself.

[defiantnerd]

Attempted in #574, merged as 7add75b, and reverted in #575 — it broke the editor in Logic. Recording what was tried so the next attempt does not repeat it.

What #574 did, and what broke

It addressed all four points here: deferred the request to onIdle(), marshalled to the main thread, made gui_can_resize() and gui_request_resize() agree, and ran the size through adjust_size.

The fourth point is where it went wrong. #574 read "run the size through adjust_size in -setFrame: before calling set_size, and honour the result" as also applying the adjusted size to the view's frame. AU Cocoa gives a host no way to hear about that. VST3 is asked checkSizeConstraint() before it lays out, and the standalone has windowWillResize:toSize:, but an AU Cocoa host simply sets the frame. A view that sizes itself away from its container is drawn out of place inside it, and AppKit's autoresizing carries the offset forward indefinitely, because it derives each new frame from the current frame rather than from the container.

Measured in Logic, which sets the view to twice its size at open and back again: the window stayed at the doubled size while the view did not, the editor was drawn out of place inside it, and a zoom change resized only the hosting service's window.

A second attempt took the narrower reading — leave the frame alone, adjust only the value handed to set_size, which is what VST3's onSize() does — and that broke the editor in Logic too.

Open question for the next attempt

Given that both readings failed, the fourth point above may be wrong for this format. Worth settling first: does adjust_size belong in the host-layout path on AU Cocoa at all, or should set_size just carry the host's size as it did before?

One hypothesis, untested and offered only as a starting point: with a plugin that clamps to a minimum size, the host lays the view out small, the wrapper reports the clamped-up size, the plugin asks for that size back, the host lays out small again — a loop. If that is what happens, adjust_size here is actively harmful and only request_resize should be adjusted.

Corrections to two claims in #574

Both verified with an AppKit probe on macOS 26, and both are worth knowing before touching this file again:

• Autoresizing calls -setFrame:, not -setFrameSize:. -setFrame: funnels through -setFrameSize:, so -setFrameSize: alone is the single choke point that catches every route. AUv2: defer request_resize to the main thread and stop reentering the plugin #574's second commit claimed the opposite and overrode both, which just ran the path twice per layout.
• [super initWithFrame:] calls neither setter.
• Separately: gui_can_resize() has no callers anywhere in the tree, so point 3 above is inert whichever way it is answered. Worth fixing for consistency, but it cannot be the cause of anything.

On verifying a fix

Nothing in this repository's CI can see an editor defect: auval never opens the editor, and the clap-first test plugin has no GUI. Both failed attempts passed auval -strict cleanly. A real host has to be in the loop before the next attempt lands.

[defiantnerd]

Second attempt: #576, which lands in next through #577 (#576 merged into its stacked base instead of next).

It answers the open question above: in AU Cocoa, adjust_size does not belong in the host's layout path, and set_size carries the host's size as before. It keeps the main-thread path exactly as it was: synchronous setFrame:(0, 0, w, h), with the set_size echo. It marshals only an off-thread request_resize to onIdle(). From #574 it keeps only the lifetime fixes.

Why that path must stay as it is, from Logic's own AU view wrapper (XAUWrapView in MAAudioUnitSupport.framework):

• It resizes Logic's window from the AU view's NSViewFrameDidChangeNotification.
• It always places the AU view at (0, 0).
• When the host sets the frame, it adopts whatever size the AU view ends up at.

#574 kept the view's origin, deferred the resize and suppressed the echo. After Logic's doubled-size layout at open, the view no longer sat at its container's origin. Later requests then either didn't reach Logic or grew the view out of place.

The loop hypothesis did not apply to the plugin we tested: its adjust_size returned what it was given, and its set_size never asks for a size.

One more finding for anyone chasing resizing in Logic: XAUWrapView's AUv2 initializer stores mIsScalable = NO / mIsUnlimited = NO as constants, and only the AUv3 path can be scalable. So an AUv2 editor gets no edge-resize from Logic. A plugin that wants resizing there needs its own grip calling request_resize.

[defiantnerd]

Resolved on next by #576/#577 (b5c8586), after #574 was reverted in #575. Two of the four points above are fixed; two are deliberately not, and should not be attempted again — both were tried and both broke the editor in Logic.

Fixed

2. Nothing marshals to the main thread — an off-thread request_resize now parks the size in an atomic and applies it from onIdle(). Main-thread is decided with [NSThread isMainThread], not Plugin::is_main_thread(), which also answers true under AlwaysMainThread().

3. The answer to the plugin is always true — request_resize now refuses a zero or >65535 size and a closed editor, and gui_can_resize() agrees with it instead of contradicting it. (Note for anyone auditing: gui_can_resize() has no callers anywhere in the tree, so it is consistency only.)

Also carried over from #574: _uiconn._window is cleared when the editor goes away, so a parked request cannot reach a freed NSView, and _uiIsOpened is atomic because request_resize reads it from any thread.

Deliberately not fixed — closing these two as wontfix

1. The plugin is reentered synchronously. On the main thread this is kept exactly as it was: the view is set to (0, 0, w, h) immediately and its -setFrame: override hands the size back through set_size() before request_resize returns.

That echo, and the (0, 0) origin, are what an AU Cocoa host depends on. Logic's AU view wrapper resizes its own window from the AU view's NSViewFrameDidChangeNotification and places the view at (0, 0) itself. #574 deferred the resize, preserved the view's current origin instead of zeroing it, and suppressed the echo — the result was a view left out of place in its container with the editor unrelaid. The reentrancy this issue describes is real, but on this format it is load-bearing.

4. -setFrame: does not ask adjust_size. Adding it is what broke Logic, in both readings:

• Applying the adjusted size to the view's frame (what AUv2: defer request_resize to the main thread and stop reentering the plugin #574 did) desynchronises the view from its container. AU Cocoa has no checkSizeConstraint() the way VST3 does and no windowWillResize:toSize: the way the standalone does, so a host cannot hear that the plugin snapped the size. AppKit's autoresizing then carries the offset forward indefinitely, since it derives each new frame from the current frame rather than from the container.
• Adjusting only the value handed to set_size, leaving the frame alone — the narrower reading, and what VST3's onSize() does — broke the editor in Logic as well.

The view class on next is therefore byte-identical to what it was before #574: no -setFrameSize: override, no adjust_size in the host's layout path.

Two corrections to claims made in #574, worth knowing before touching this file

Verified with an AppKit probe on macOS 26:

• Autoresizing calls -setFrame:, not -setFrameSize:. -setFrame: funnels through -setFrameSize:, so -setFrameSize: is the single choke point that catches every route. AUv2: defer request_resize to the main thread and stop reentering the plugin #574 claimed the opposite and overrode both, which only ran the whole path twice per layout.
• [super initWithFrame:] calls neither setter.
• Also: NSViewFrameDidChangeNotification is posted synchronously from inside [super setFrameSize:], and is not posted at all when the size is unchanged. Any scheme that brackets a frame change with a flag will swallow a host that answers from within that notification.

On verifying changes here

Nothing in this repository's CI can see an editor defect: auval never opens the editor, and the clap-first test plugin has no GUI. Both reverted attempts passed auval -strict cleanly. Changes to the AUv2 editor path need a real host before they land, not after.