Skip to content

docs(skills): document one-shot observe CLI workflows#39

Open
quanruzhuoxiu wants to merge 1 commit into
web-infra-dev:mainfrom
quanruzhuoxiu:agent/observe-cli-guidance
Open

docs(skills): document one-shot observe CLI workflows#39
quanruzhuoxiu wants to merge 1 commit into
web-infra-dev:mainfrom
quanruzhuoxiu:agent/observe-cli-guidance

Conversation

@quanruzhuoxiu

Copy link
Copy Markdown
Contributor

Summary

  • document the one-shot observe command for the browser, Android, iOS, HarmonyOS, and computer skills
  • explain when to use observe instead of a current-state assert
  • include platform-specific connection, device, and continuous-frame-source guidance
  • update each skill workflow so transient UI verification stays within one CLI invocation

Motivation

Transient UI such as toasts, banners, animations, and screen transitions may disappear before a separate assert command runs. These updates teach agents to use the new one-shot CLI observation workflow introduced by web-infra-dev/midscene#2848.

Impact

Agents can verify transient UI without attempting to split startObserving and stop across stateless CLI processes. Ordinary assert remains the recommended command for current or final screen state.

Validation

  • git diff --check
  • verified the frontmatter and command examples in all five modified skill files
  • validated the underlying Web CLI against the transient-toast fixture with --interval-ms 200, --max-frames 30, and --watchdog-ms 15000; the observed-window assertion passed

@quanru
quanru marked this pull request as ready for review July 23, 2026 07:01

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 22e7955511

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/browser/SKILL.md
Use `observe` when the state you need to verify may appear and disappear while an action is running, such as a toast, loading banner, animation, or screen transition. It starts observation, runs the action, stops observation, and checks the assertion in one CLI invocation:

```bash
npx -y @midscene/web@1 observe \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Delay documenting unreleased observe command

This and the equivalent blocks added to the Android/iOS/HarmonyOS/computer skills point agents at @midscene/*@1 observe, but the supporting Midscene change is not in a released @1 package yet: I checked the referenced upstream PR web-infra-dev/midscene#2848 and it is still open, while current Midscene main only registers the common CLI tools take_screenshot, act, and assert. If these skill docs are published before that CLI release, any transient-UI workflow will choose observe and fail with an unknown command; please gate this guidance on the release or pin it to a version that contains the command.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant