Add AI-assisted triage by renanrodrigo · Pull Request #3 · ubuntu/startriage

renanrodrigo · 2026-06-16T11:59:32Z

This adds the whole AI-assisted functionality to startriage. It's full of abstractions and details but it works well.

Vibe coded
--ai for usual triage second opinion
ai-triage for single bugs

Also make the agent perma prompt part of the package

- make it part of the markdown when that is used - save to separate file if not used - print to stdout for ai-triage command

grayson-wolf

looks good, to me, just a few suggestions / nit-questions

TheJJ

coming together pretty nicely. the overall architecture is good, i'd say.

one thing that worries me is the implicit assumption this is run in a secure container now:
how about a text-only analysis without funny containers as the first agent, and when it decides we wanna run shell stuff, it spins up a subagent in an lxd container (using the same mechanism we use for git ubuntu deltarebase resolving - we can hopefully create a shared ai lxd containering library so we don't have to duplicate/reinvent stuff)

otherwise many nits and naming issues, you'll see. great work, overall :)

Accept only a bare/#-prefixed integer or a genuine Launchpad bug URL (.../+bug/<n> or /bugs/<n>) and raise ValueError otherwise.

Plain 'analyze <bug>' now prints the bug metadata (status, tags, affected targets, description, comments) with no provider or credentials required; 'analyze --ai <bug>' runs the agent as the old 'ai-triage' command did.

triage --ai no longer writes a separate autotriage-YYYY-MM-DD.md file. Without --markdown the AI section is printed to stdout after the normal triage output; with --markdown it is folded into that same file behind the review notice, giving one cohesive report. Drops the now-unused report_filename/write_report/resolve_report_dir helpers and their tests, and updates the prompt and README accordingly.

renanrodrigo · 2026-07-03T02:22:09Z

@TheJJ I addressed most of your review points, responded to a couple inline.

I see your point with the two-step agent run, but I would ask you if we could not do this in a future iteration. You mentioned there may be some overlap in the "Run this container and do stuff" tasks - let's go for the shared LXD library then. But the size and scope of this PR make me thing later is better.

For now we can tell users that this is potentially unsafe, and ask them to please containerize (or isolate the environment right) before running AI commands.

WDYT?

TheJJ

just minor things left, i think for a first version we are nearly ready.

this runs the ai analysis within the snap confinement, but as future work we should add properly containerized ai execution - maybe open an issue for that then :)

TheJJ · 2026-07-03T12:51:43Z

+def _build_ai_provider(config: StarTriageConfig) -> Provider | None:
+    """Build the AI provider, printing a friendly hint and returning None on misconfig."""
+    from .ai import build_provider
+
+    try:
+        return build_provider(config.ai)
+    except AIConfigError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return None


shouldn't be part of cli.py, rather ai/provider.py. return value should always be Provider, and if that fails, let the AIConfiError be raised to top for crash.

TheJJ · 2026-07-03T12:52:07Z

+def _emit_ai_report(report: str, markdown_path: Path | None) -> None:
+    """Emit an AI ``report`` for a ``triage --ai`` run.
+
+    With ``--markdown`` the AI section is appended (behind a review notice) to
+    that same file, so the human triage report and the AI aid live in one
+    cohesive document. Without ``--markdown`` the report is printed to stdout,
+    together with the normal triage output that already went there.
+    """
+    from .ai import append_report
+    from .ai.render import AI_APPEND_NOTICE
+
+    if markdown_path is not None:
+        append_report(markdown_path, report)
+        print(f"AI triage appended to {markdown_path}")
+    else:
+        print("\n" + AI_APPEND_NOTICE + report)


that should be in ai/render.py

TheJJ · 2026-07-03T12:57:35Z

+            return self.github_token or _first_env(COPILOT_TOKEN_ENV_VARS)
+        return self.openrouter_api_key or _first_env(OPENROUTER_KEY_ENV_VARS)
+
+    def require_configured(self) -> None:


@model_validator(mode='after') def check_token(self) -> Self: if self.resolve_token() is not None: return self match self.provider: case AIProvider.copilot: ... raise ValueError('bla') case _: raise RuntimeError("unhandled provider")

Move the missing-credential check from AIConfig.require_configured() onto a model validator that only enforces when validated with a require_ai context. This keeps the [ai] section optional for non-AI commands while build_provider re-validates with that context so a misconfigured provider fails before a session starts. AIConfigError is not a ValueError, so pydantic propagates it unwrapped to the top.

renanrodrigo · 2026-07-03T14:35:08Z

@TheJJ issue open

Claude is worried that the model validator will break users who have no [ai] configured when trying to run non-ai commands, so it proposed something in-the-middle.

TheJJ

great, now the only missing piece is the discussed --yolo flag to grant/deny execution rights to the agent.

to implement it, we could turn --ai into having a parameter.

--ai=deny (Default)
--ai=always
--ai=ask

and this flag could then be in a parent parser, as mentioned below.

--ai now takes a required level (restricted/full/ask) instead of being a plain on/off switch, and lives on a parent parser shared by triage and analyze. The level is mandatory so --ai never swallows a following bug number as its value, keeping 'analyze --ai 123' unambiguous (it rejects 123 as an invalid level). A new AIPermission enum carries the level through to build_provider.

build_provider takes the permission level and threads it to CopilotProvider, which selects the session's on_permission_request handler: full auto-approves every tool, restricted rejects them all (and appends a text-only note to the prompt), ask prompts on the terminal per call.

renanrodrigo · 2026-07-08T03:22:19Z

Claude iteration done
I just changed the plan to:
a) for clarity, make the permission level mandatory on the CLI
b) changed the level to restricted, full, ask

Then fire the cannons and here we are

TheJJ

great, for a first version we can merge and test this for further improvements.

renanrodrigo added 11 commits June 15, 2026 22:29

Add configuration options for the AI-assisted triage

3c5d04f

Add custom to_agent_payload method

ff1d8de

Add provider layer to copilot+openrouter

008d883

Add agent loop and result contract for the AI-assisted triage

3cb5ec6

Also make the agent perma prompt part of the package

Render triage results into the autotriage report

14996da

Add CLI --ai option and ai-triage command

940728a

Package the Copilot runtime and ubuntu-dev-tools

ac1c44a

Add step logging for AI triage observability

3f20940

Remove timeouts and add a spinner to follow AI triage execution

4f1acf2

Reformat AI generated output

5851c2f

- make it part of the markdown when that is used - save to separate file if not used - print to stdout for ai-triage command

Reformat agent_prompts into proper markdown

7041f85

grayson-wolf suggested changes Jun 16, 2026

View reviewed changes

Comment thread startriage/data/agents_prompt.md Outdated

Comment thread startriage/data/defaults.toml

Comment thread startriage/data/agents_prompt.md Outdated

TheJJ requested changes Jun 26, 2026

View reviewed changes

renanrodrigo added 12 commits July 2, 2026 16:37

ai: log full backtrace when a provider run fails

2687096

ai: use match statements for provider kwargs dispatch

97729f2

tests: fix indentation of inline TOML heredocs

9091f39

spinner: make TTY detection a property of Spinner

d701770

ai: resolve report directory before running the agent

991915a

tests: drop test_packaging

521641f

ai: parse bug numbers strictly

7c17019

Accept only a bare/#-prefixed integer or a genuine Launchpad bug URL (.../+bug/<n> or /bugs/<n>) and raise ValueError otherwise.

cli: use 'is not None' for AI config-set flags

f3877ad

cli: move AI orchestration into ai/

77fed27

cli: rename ai-triage to 'analyze' with an --ai flag

c1f64d8

Plain 'analyze <bug>' now prints the bug metadata (status, tags, affected targets, description, comments) with no provider or credentials required; 'analyze --ai <bug>' runs the agent as the old 'ai-triage' command did.

prompt: improve prompt instructions

361ef7a

TheJJ requested changes Jul 3, 2026

View reviewed changes

renanrodrigo mentioned this pull request Jul 3, 2026

AI-triage should have separate steps for analyzing and executing code #4

Open

renanrodrigo added 2 commits July 3, 2026 11:32

cli: move ai provider build and report emit into the ai package

d1a76d2

TheJJ requested changes Jul 6, 2026

View reviewed changes

Comment thread startriage/cli.py Outdated

Comment thread startriage/ai/provider.py Outdated

renanrodrigo added 3 commits July 8, 2026 00:18

docs: document the --ai permission levels and the execution warning

34870d9

TheJJ approved these changes Jul 8, 2026

View reviewed changes

TheJJ merged commit 0796cf8 into ubuntu:main Jul 8, 2026

renanrodrigo deleted the ai-triage branch July 8, 2026 12:42

Uh oh!

Conversation

renanrodrigo commented Jun 16, 2026

Uh oh!

grayson-wolf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TheJJ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

renanrodrigo commented Jul 3, 2026

Uh oh!

TheJJ left a comment

Choose a reason for hiding this comment

Uh oh!

TheJJ Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

TheJJ Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

TheJJ Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

renanrodrigo commented Jul 3, 2026

Uh oh!

TheJJ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

renanrodrigo commented Jul 8, 2026

Uh oh!

TheJJ left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants