Skip to content

Commit 96f7576

Browse files
cynarlabclaude
andcommitted
test(dart): narrow the Android e2e helper to what the Flutter tool does not already do (#790)
From the scoped review of #883: - `adb forward --remove-all` is host-wide (killforward-all drops every device's listeners), so it took other tools' forwards with it. The helper now removes only this emulator's entries from `adb forward --list`. - `flutter run` force-stops, builds and installs on every launch (android_device.dart startApp → stopApp, installApp), and `flutter build apk --debug` builds three ABIs while `flutter run` builds the device's, so the warm-up used to be replaced by the first launch. The prebuild now targets the device's ABI (ro.product.cpu.abi); the docstring, guide, KNOWN_ISSUES and spike notes say what it buys — a warm Gradle cache and the first install — instead of describing it as a fix, and the once-seen "ended without a stop" is labelled as seen once. - `adb` errors now carry stderr (install failures report their reason), and an incompatible installed APK is uninstalled once and retried, the way flutter_tools does. - adb discovery covers Android Studio's default SDK paths on macOS and Linux; the emulator gate consults Flutter's device list before starting an adb server. - Both Flutter suites wait for a stop in `wait_for_stop` slices of at most 45 s: the MCP client caps a request at 60 s, and one long server-side wait failed on the client side when a build kept the first stop away for 63 s (seen on the emulator's integration case). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 parent 1c4738e commit 96f7576

7 files changed

Lines changed: 117 additions & 41 deletions

File tree

‎docs/KNOWN_ISSUES.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -114,10 +114,14 @@ Dart 3.13 and Flutter 3.47 — and none of them is a session error.
114114
"Connecting to the VM Service is taking longer than expected" (measured, 60 s). The same
115115
recipe works for desktop apps and Dart programs. Launch the app through mcp-debugger instead;
116116
#882 tracks the `--device-vmservice-port` / `--host-vmservice-port` / `--no-dds` measurements.
117-
- **On an Android emulator, stop the previous app instance before the next launch.** A live
118-
instance being replaced by the next install, or a stale adb port forward, made a launch hang
119-
or end without a stop (measured). `adb shell am force-stop <package>` and
120-
`adb forward --remove-all` first; the e2e helper does both, and installs the APK up front.
117+
- **On an Android emulator, the first launch after a fresh install may end without a stop.**
118+
Seen once: the first `flutter run` right after the debug APK was first installed ended with
119+
"The program ended without reporting an exit code" instead of pausing; the next launch was
120+
fine. `flutter run` stops and reinstalls the app itself on every launch, so there is nothing
121+
to do by hand; relaunch. The e2e helper warms the Gradle build and does the first install
122+
before the timed cases. A cold Gradle build on a fresh machine (minutes) belongs to the first
123+
build, not to the debugger: the launch answers `pending: true` and `wait_for_stop` collects
124+
the stop.
121125
- **`dart test` fails under `%LOCALAPPDATA%\Temp` with the winget SDK (Windows).**
122126
package:test spawns its frontend server through a cwd-relative SDK path that does not
123127
resolve from there (`The system cannot find the file specified … dartaotruntime.exe`).

‎docs/dart/README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -138,11 +138,11 @@ Android emulator (`deviceId: "emulator-5554"`, the AVD booted first; measured th
138138
`tests/e2e/mcp-server-smoke-flutter-android.test.ts`, Gradle warm): a `build()` breakpoint under
139139
`flutter run` in ~21 s, an `integration_test` breakpoint in ~28 s with the app's breakpoint hit by
140140
the test's tap right after; a cold Gradle build on a fresh machine takes minutes and belongs to
141-
the first build, not to the debugger. Stop the previous app instance (`adb shell am force-stop
142-
<package>`) and clear forwards (`adb forward --remove-all`) before launching again; the e2e helper
143-
does so, and installs the debug APK up front because the first run after a fresh install once
144-
ended without stopping. Attaching to an app another `flutter run` started on the emulator does
145-
not connect yet (#882; see `docs/KNOWN_ISSUES.md`).
141+
the first build, not to the debugger. `flutter run` stops and reinstalls the app itself on every
142+
launch; the e2e helper only warms the build for the device's ABI and does the first install
143+
ahead of the timed cases, because the first run right after a fresh install once ended without
144+
stopping (relaunch if that happens). Attaching to an app another `flutter run` started on the
145+
emulator does not connect yet (#882; see `docs/KNOWN_ISSUES.md`).
146146

147147
## Troubleshooting
148148

‎docs/dart/spike-notes.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -355,10 +355,10 @@ Measured through `dist/index.js` with `tests/e2e/mcp-server-smoke-flutter-androi
355355
| Case | Result |
356356
|---|---|
357357
| `flutter run -d emulator-5554`, breakpoint in `build()` | `pending: true` at 2.5 s; the breakpoint at **21.3 s**; `_ProbeAppState.build`, `counter` → `0`; `continue` keeps it running; `close_debug_session` terminates the app |
358-
| the same, first launch right after a fresh APK install | ended at 27.9 s with "The program ended without reporting an exit code" and no stop (once; the next launch was fine) — hence the up-front install and the force-stop before every launch |
358+
| the same, first launch right after a fresh APK install | ended at 27.9 s with "The program ended without reporting an exit code" and no stop (once; the next launch was fine) — hence the helper's warm prebuild for the device's ABI and first install ahead of the timed cases; `flutter run` itself stops and reinstalls the app every time |
359359
| `integration_test` on the emulator | test breakpoint at **28.3 s**, the app's `increment()` breakpoint at **28.7 s** on the test's tap, `counter` → `0`, `✓ device increment`, end `stopped` at 31.6 s |
360360

361361
- `flutter build apk --debug` on the fresh example with warm Gradle caches: **45 s**.
362-
- Before each launch: `adb shell am force-stop com.example.flutter_probe`, `adb forward --remove-all`.
362+
- Before each launch the helper force-stops a leftover instance and removes this emulator's own adb forwards (from `adb forward --list`; `adb forward --remove-all` is host-wide). Whether a stale forward ever caused the "device offline" failure is not established — the force-stop may be all that mattered.
363363
- The proxy log records output events by shape only (issue #852): to read the device's output
364364
after the fact, `get_output` before the session closes.

‎tests/e2e/README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,8 +88,9 @@ This directory contains end-to-end smoke tests that verify the MCP debugger serv
8888
- `mcp-server-smoke-flutter-android.test.ts` runs `flutter run` and the `integration_test` on a
8989
booted Android emulator (`flutter emulators --launch <id>` is the opt-in; it self-skips without
9090
Flutter, adb or an online emulator, and with `MCP_SKIP_FLUTTER_ANDROID=1`). The helper generates
91-
`android/`, builds the debug APK once, installs it, and stops the app and clears adb forwards
92-
before each launch. Never runs on CI
91+
`android/`, builds the debug APK once for the device's ABI and installs it (a warm build; `flutter
92+
run` stops and reinstalls on its own anyway), and before each launch stops a leftover instance
93+
and removes that emulator's own adb forwards. Never runs on CI
9394

9495
### 13. `mcp-server-smoke-javascript-sse.test.ts`
9596
- Tests JavaScript adapter over SSE transport

‎tests/e2e/dart-example-utils.ts‎

Lines changed: 54 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -214,18 +214,27 @@ export function adbPath(): string | null {
214214
}
215215
const onPath = which.sync('adb', { nothrow: true });
216216
if (onPath) return onPath;
217-
if (process.platform === 'win32' && process.env.LOCALAPPDATA) {
218-
const candidate = path.join(process.env.LOCALAPPDATA, 'Android', 'Sdk', 'platform-tools', exe);
217+
// Android Studio's default SDK locations, the ones flutter_tools probes too.
218+
const home = process.env.HOME ?? process.env.USERPROFILE ?? '';
219+
const defaults = process.platform === 'win32'
220+
? [path.join(process.env.LOCALAPPDATA ?? '', 'Android', 'Sdk')]
221+
: process.platform === 'darwin'
222+
? [path.join(home, 'Library', 'Android', 'sdk')]
223+
: [path.join(home, 'Android', 'Sdk')];
224+
for (const sdk of defaults) {
225+
const candidate = path.join(sdk, 'platform-tools', exe);
219226
if (existsSync(candidate)) return candidate;
220227
}
221228
return null;
222229
}
223230

224-
function adb(args: string[], timeoutMs = 120_000): { status: number | null; stdout: string } {
231+
function adb(args: string[], timeoutMs = 120_000): { status: number | null; stdout: string; out: string } {
225232
const exe = adbPath();
226-
if (!exe) return { status: null, stdout: '' };
233+
if (!exe) return { status: null, stdout: '', out: '' };
227234
const r = spawnSync(exe, args, { encoding: 'utf8', windowsHide: true, timeout: timeoutMs });
228-
return { status: r.status, stdout: r.stdout ?? '' };
235+
const stdout = r.stdout ?? '';
236+
// adb reports install failures on stderr (`adb: failed to install …`).
237+
return { status: r.status, stdout, out: `${stdout}${r.stderr ?? ''}` };
229238
}
230239

231240
let cachedEmulator: string | null | undefined;
@@ -238,25 +247,47 @@ let cachedEmulator: string | null | undefined;
238247
export function flutterEmulatorDeviceId(): string | null {
239248
if (cachedEmulator !== undefined) return cachedEmulator;
240249
if (process.env.MCP_SKIP_FLUTTER_ANDROID === '1' || !adbPath()) return (cachedEmulator = null);
250+
// Flutter's own list first: no emulator there means no adb server to start for nothing.
251+
const emulators = flutterDeviceIds().filter((id) => id.startsWith('emulator-'));
252+
if (emulators.length === 0) return (cachedEmulator = null);
241253
const online = new Set(
242254
adb(['devices']).stdout.split(/\r?\n/).map((l) => l.trim().split(/\s+/)).filter((p) => p.length === 2 && p[1] === 'device').map((p) => p[0])
243255
);
244-
cachedEmulator = flutterDeviceIds().find((id) => id.startsWith('emulator-') && online.has(id)) ?? null;
256+
cachedEmulator = emulators.find((id) => online.has(id)) ?? null;
245257
return cachedEmulator;
246258
}
247259

248-
/** Stop the probe app and drop stale port forwards: a live previous instance or a stale forward makes the next launch hang (measured). */
260+
/**
261+
* Before a launch: stop the probe app and drop the port forwards that belong to this emulator.
262+
* `flutter run` force-stops and reinstalls on its own (android_device.dart `startApp`), so the
263+
* stop only matters for an instance a previous test left behind; the forwards are removed one by
264+
* one from `adb forward --list` because `adb forward --remove-all` is host-wide and would take
265+
* other tools' forwards with it.
266+
*/
249267
export function resetAndroidApp(deviceId: string): void {
250268
adb(['-s', deviceId, 'shell', 'am', 'force-stop', FLUTTER_PROBE_PACKAGE], 30_000);
251-
adb(['forward', '--remove-all'], 30_000);
269+
for (const line of adb(['forward', '--list'], 30_000).stdout.split(/\r?\n/)) {
270+
const [serial, local] = line.trim().split(/\s+/);
271+
if (serial === deviceId && local) adb(['-s', deviceId, 'forward', '--remove', local], 30_000);
272+
}
252273
}
253274

254275
let androidPrepared = false;
255276

277+
/** The `--target-platform` matching the device's ABI, so the warm-up builds what `flutter run` will run. */
278+
function androidTargetPlatform(deviceId: string): string {
279+
const abi = adb(['-s', deviceId, 'shell', 'getprop', 'ro.product.cpu.abi'], 30_000).stdout.trim();
280+
if (abi.startsWith('arm64')) return 'android-arm64';
281+
if (abi.startsWith('armeabi')) return 'android-arm';
282+
return 'android-x64';
283+
}
284+
256285
/**
257-
* Generate the probe's `android/` folder, build the debug APK once and install it on the device,
258-
* so the launches under test never pay the first install (the first `flutter run` after a fresh
259-
* install ended without a stop once, measured; the next one was fine). Warm Gradle: ~45 s.
286+
* Generate the probe's `android/` folder, build the debug APK for the device's ABI once and
287+
* install it. `flutter run` stops, builds and installs on its own every time, so what this buys
288+
* the timed launches is a warm Gradle and dependency cache (the spike measured 425 s cold, 6–45 s
289+
* warm) and the first-ever `pm install` out of the way; the first `flutter run` after a fresh
290+
* install was seen to end without a stop once. A stale `android/` is not regenerated: delete it.
260291
*/
261292
export function prepareFlutterAndroid(deviceId: string): void {
262293
if (androidPrepared) return;
@@ -267,11 +298,19 @@ export function prepareFlutterAndroid(deviceId: string): void {
267298
const r = runFlutter(root, ['create', '--platforms=android', '--project-name', 'flutter_probe', '.'], FLUTTER_PROBE_DIR);
268299
if (r.status !== 0) throw new Error(`flutter create --platforms=android failed: ${r.out}`);
269300
}
270-
const built = runFlutter(root, ['build', 'apk', '--debug'], FLUTTER_PROBE_DIR, 900_000);
271-
if (built.status !== 0) throw new Error(`flutter build apk --debug failed: ${built.out}`);
301+
const built = runFlutter(root, ['build', 'apk', '--debug', '--target-platform', androidTargetPlatform(deviceId)], FLUTTER_PROBE_DIR, 900_000);
302+
if (built.status !== 0) {
303+
throw new Error(`flutter build apk --debug failed (a stale examples/dart/flutter_probe/android can be deleted and regenerated): ${built.out}`);
304+
}
272305
const apk = path.join(FLUTTER_PROBE_DIR, 'build', 'app', 'outputs', 'flutter-apk', 'app-debug.apk');
273-
const installed = adb(['-s', deviceId, 'install', '-r', '-t', apk], 180_000);
274-
if (installed.status !== 0) throw new Error(`adb install failed: ${installed.stdout}`);
306+
let installed = adb(['-s', deviceId, 'install', '-r', '-t', apk], 180_000);
307+
if (installed.status !== 0) {
308+
// An older signature or version on the device (INSTALL_FAILED_UPDATE_INCOMPATIBLE): the way
309+
// flutter_tools handles it too — uninstall once and retry.
310+
adb(['-s', deviceId, 'uninstall', FLUTTER_PROBE_PACKAGE], 60_000);
311+
installed = adb(['-s', deviceId, 'install', '-r', '-t', apk], 180_000);
312+
if (installed.status !== 0) throw new Error(`adb install failed: ${installed.out}`);
313+
}
275314
androidPrepared = true;
276315
}
277316

‎tests/e2e/mcp-server-smoke-flutter-android.test.ts‎

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@
33
* emulator-NNNN` with a breakpoint in `build()`, and an `integration_test` on the emulator whose
44
* tap hits the app's breakpoint. Measured through the real server on 2026-10-09 (Flutter 3.47.7,
55
* Windows, AVD `mcp_api35` booted headless, Gradle warm): the `build()` breakpoint in ~21 s, the
6-
* integration test's breakpoints in ~28 s; the first `flutter run` after a fresh APK install once
7-
* ended without a stop, so the helper installs the APK up front and stops the app and clears
8-
* port forwards before every launch.
6+
* integration test's breakpoints in ~28 s. `flutter run` stops, builds and installs the app on
7+
* its own every launch; the helper warms the Gradle build for the device's ABI and does the
8+
* first install ahead of the timed cases (the first run right after a fresh install once ended
9+
* without a stop), and before each launch stops a leftover instance and removes this emulator's
10+
* own adb forwards.
911
*
1012
* Opt-in by booting an emulator (`flutter emulators --launch <id>`); self-skips without Flutter,
1113
* adb, an online emulator, or with `MCP_SKIP_FLUTTER_ANDROID=1`. CI never runs it.
@@ -112,11 +114,27 @@ describe.skipIf(!EMULATOR)(`MCP Server Flutter Android Smoke Test @requires-flut
112114
return res;
113115
}
114116

117+
/**
118+
* `wait_for_stop` in slices under the MCP client's 60 s request cap: a Gradle build can keep
119+
* the first stop away for longer than that (seen: 63 s), and a single long server-side wait
120+
* then fails on the client side, not the debugger's.
121+
*/
122+
async function waitForPause(timeoutMs: number, what: string) {
123+
const deadline = Date.now() + timeoutMs;
124+
for (;;) {
125+
const slice = Math.min(45_000, Math.max(1_000, deadline - Date.now()));
126+
const waited = await call('wait_for_stop', { timeout: slice });
127+
if (waited.state === 'paused') return (await getSession())!;
128+
const stillRunning = waited.pending === true || waited.state === 'running';
129+
if (!stillRunning || Date.now() >= deadline) {
130+
throw new Error(`${what}: wait_for_stop answered ${JSON.stringify(waited)}; output tail: ${(await output()).slice(-600)}`);
131+
}
132+
}
133+
}
134+
115135
async function firstStop(launched: Record<string, unknown>, timeoutMs: number) {
116136
if (launched.state === 'paused') return (await getSession())!;
117-
const waited = await call('wait_for_stop', { timeout: timeoutMs });
118-
expect(waited.state, `first stop (wait_for_stop answered ${JSON.stringify(waited)}; output: ${(await output()).slice(-600)})`).toBe('paused');
119-
return (await getSession())!;
137+
return waitForPause(timeoutMs, 'first stop');
120138
}
121139

122140
async function createSession(name: string) {
@@ -160,8 +178,7 @@ describe.skipIf(!EMULATOR)(`MCP Server Flutter Android Smoke Test @requires-flut
160178
expect((await frames())[0].line).toBe(testLine);
161179

162180
await callToolSafely(mcpClient!, 'continue_execution', { sessionId });
163-
const atApp = await call('wait_for_stop', { timeout: 120000 });
164-
expect(atApp.state, JSON.stringify(atApp)).toBe('paused');
181+
await waitForPause(120000, "the app's breakpoint on the test's tap");
165182
const top = (await frames())[0];
166183
expect(top.file?.toLowerCase()).toBe(FLUTTER_EXAMPLES.main.toLowerCase());
167184
expect(top.line).toBe(appLine);

‎tests/e2e/mcp-server-smoke-flutter.test.ts‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -120,12 +120,28 @@ describe.skipIf(SKIP_FLUTTER)('MCP Server Flutter Debugging Smoke Test @requires
120120
return res;
121121
}
122122

123+
/**
124+
* `wait_for_stop` in slices under the MCP client's 60 s request cap: a cold compile or a
125+
* runner build can keep the first stop away for longer than that, and a single long
126+
* server-side wait then fails on the client side, not the debugger's.
127+
*/
128+
async function waitForPause(timeoutMs: number, what: string) {
129+
const deadline = Date.now() + timeoutMs;
130+
for (;;) {
131+
const slice = Math.min(45_000, Math.max(1_000, deadline - Date.now()));
132+
const waited = await call('wait_for_stop', { timeout: slice });
133+
if (waited.state === 'paused') return (await getSession())!;
134+
const stillRunning = waited.pending === true || waited.state === 'running';
135+
if (!stillRunning || Date.now() >= deadline) {
136+
throw new Error(`${what}: wait_for_stop answered ${JSON.stringify(waited)}; output tail: ${(await output()).slice(-600)}`);
137+
}
138+
}
139+
}
140+
123141
/** The launch answers `pending` while the test VM compiles; the first stop is what matters. */
124142
async function firstStop(launched: Record<string, unknown>, timeoutMs: number) {
125143
if (launched.state === 'paused') return (await getSession())!;
126-
const waited = await call('wait_for_stop', { timeout: timeoutMs });
127-
expect(waited.state, `first stop (wait_for_stop answered ${JSON.stringify(waited)})`).toBe('paused');
128-
return (await getSession())!;
144+
return waitForPause(timeoutMs, 'first stop');
129145
}
130146

131147
async function createSession(name: string) {
@@ -219,8 +235,7 @@ describe.skipIf(SKIP_FLUTTER)('MCP Server Flutter Debugging Smoke Test @requires
219235
expect((await frames())[0].line).toBe(testLine);
220236

221237
await callToolSafely(mcpClient!, 'continue_execution', { sessionId });
222-
const atApp = await call('wait_for_stop', { timeout: 60000 });
223-
expect(atApp.state, JSON.stringify(atApp)).toBe('paused');
238+
await waitForPause(60000, "the app's breakpoint on the test's tap");
224239
const top = (await frames())[0];
225240
expect(top.file?.toLowerCase()).toBe(FLUTTER_EXAMPLES.main.toLowerCase());
226241
expect(top.line).toBe(appLine);

0 commit comments

Comments
 (0)