Skip to content

Add IME-safe enter-to-submit guide and evals - #991

Merged
rviscomi merged 5 commits into
mainfrom
ime-submit
Aug 26, 2026
Merged

Add IME-safe enter-to-submit guide and evals#991
rviscomi merged 5 commits into
mainfrom
ime-submit

Conversation

@rviscomi

@rviscomi rviscomi commented Jul 6, 2026

Copy link
Copy Markdown
Member

Addresses #979

  • creates a new ime-safe-enter-submit guide
  • adds a reference to the guide from the Forms skill

@micahjo7

micahjo7 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator
image

ran evals against this, looks like on Codex it is failing to retrieve the right guide

on Claude Code and Jetski CLI the results looks like this:

image

maybe the task could use some tweaking?

@rviscomi

rviscomi commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Thanks @micahjo7. I seem to be blocked from installing Codex (haven't had to run it before) so I'm unable to eval it locally. Is there a change you think would help it pick up the guide?

@micahjo7

micahjo7 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

I discovered this was a local eval issue- basically, Codex CLI in our eval harness does not inheriting PATH, meaning the npx search/retrieve calls were using the published version of the package rather than the local version. So when these evals ran, it did not have the guide in this CL within the CLI tool. After fixing that, the evals look good (consistent with the claude/jetski results above). Sorry for the false alarm!

Will work on a fix for the issue.

@rviscomi

rviscomi commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Ah ok great, thanks for confirming!

@rviscomi
rviscomi requested review from LeaVerou and removed request for malchata and philipwalton July 22, 2026 21:28

@LeaVerou LeaVerou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM though it's not an API I have deep expertise on. Might want to link to the Safari bug so the agent can verify it's still current (yes, we'd update the guide, but 💩 happens) and to avoid wasting tokens hunting it down itself.

@rviscomi
rviscomi enabled auto-merge (squash) August 26, 2026 14:26
@rviscomi
rviscomi merged commit 794acf9 into main Aug 26, 2026
11 checks passed
@rviscomi
rviscomi deleted the ime-submit branch August 26, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants