Skip to content

Add IME-safe enter-to-submit guide and evals#991

Open
rviscomi wants to merge 2 commits into
mainfrom
ime-submit
Open

Add IME-safe enter-to-submit guide and evals#991
rviscomi wants to merge 2 commits into
mainfrom
ime-submit

Conversation

@rviscomi

@rviscomi rviscomi commented Jul 6, 2026

Copy link
Copy Markdown
Member

Addresses #979

  • creates a new ime-safe-enter-submit guide
  • adds a reference to the guide from the Forms skill

@micahjo7

micahjo7 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator
image

ran evals against this, looks like on Codex it is failing to retrieve the right guide

on Claude Code and Jetski CLI the results looks like this:

image

maybe the task could use some tweaking?

@rviscomi

rviscomi commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Thanks @micahjo7. I seem to be blocked from installing Codex (haven't had to run it before) so I'm unable to eval it locally. Is there a change you think would help it pick up the guide?

@micahjo7

micahjo7 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

I discovered this was a local eval issue- basically, Codex CLI in our eval harness does not inheriting PATH, meaning the npx search/retrieve calls were using the published version of the package rather than the local version. So when these evals ran, it did not have the guide in this CL within the CLI tool. After fixing that, the evals look good (consistent with the claude/jetski results above). Sorry for the false alarm!

Will work on a fix for the issue.

@rviscomi

rviscomi commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Ah ok great, thanks for confirming!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants