Skip to content
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
9046e47
test: add queue-driven fake-codex harness for multi-turn tests
kingdoooo May 17, 2026
0e9b8e5
test: harden fake-codex harness close() and dedupe saves
kingdoooo May 17, 2026
3c47c59
feat(adversarial-review): add investigate + finalize prompt templates
kingdoooo May 17, 2026
53f9e75
feat(codex): add runAppServerInvestigation for two-phase reviews
kingdoooo May 17, 2026
2708672
test: strengthen soft-error assertion to check error message
kingdoooo May 17, 2026
fbb2bc7
feat(adversarial-review): add investigate/finalize prompt builders
kingdoooo May 17, 2026
cfa0423
fix(adversarial-review): route self-collect through two-phase investi…
kingdoooo May 17, 2026
8b25e7b
feat(adversarial-review): show banner when investigation is truncated
kingdoooo May 17, 2026
93609a5
feat(adversarial-review): plumb --max-investigation-turns CLI flag
kingdoooo May 17, 2026
a32d1d1
docs(adversarial-review): document --max-investigation-turns
kingdoooo May 17, 2026
5b767ff
test(commands): allow --max-investigation-turns in argument-hint asse…
kingdoooo May 17, 2026
950329c
fix(codex): normalize runAppServerInvestigation status to numeric
kingdoooo May 17, 2026
0981f6d
test(investigation): cover phase-2 finalize transport error path
kingdoooo May 17, 2026
6992596
feat(adversarial-review): point to --max-investigation-turns in banner
kingdoooo May 17, 2026
3de1895
fix(adversarial-review): tighten --max-investigation-turns validation
kingdoooo May 18, 2026
daac52f
fix(adversarial-review): converge on any 0-command turn with an agent…
kingdoooo May 18, 2026
a7a90c7
fix(adversarial-review): retry finalize turn if the model runs commands
kingdoooo May 18, 2026
324836e
fix(adversarial-review): narrow inline-diff path to single-file reviews
kingdoooo May 18, 2026
d222542
fix(adversarial-review): force status=1 when investigation turn errors
kingdoooo May 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion plugins/codex/commands/adversarial-review.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
description: Run a Codex review that challenges the implementation approach and design choices
argument-hint: '[--wait|--background] [--base <ref>] [--scope auto|working-tree|branch] [focus ...]'
argument-hint: '[--wait|--background] [--base <ref>] [--scope auto|working-tree|branch] [--max-investigation-turns N] [focus ...]'
disable-model-invocation: true
allowed-tools: Read, Glob, Grep, Bash(node:*), Bash(git:*), AskUserQuestion
---
Expand Down Expand Up @@ -43,6 +43,7 @@ Argument handling:
- It supports working-tree review, branch review, and `--base <ref>`.
- It does not support `--scope staged` or `--scope unstaged`.
- Unlike `/codex:review`, it can still take extra focus text after the flags.
- For very large diffs that exceed the inline threshold, Codex investigates the diff with read-only commands across multiple turns. Use `--max-investigation-turns N` (default 10) to raise or lower the cap.

Foreground flow:
- Run:
Expand Down
53 changes: 53 additions & 0 deletions plugins/codex/prompts/adversarial-review-finalize.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<role>
You have just completed an investigation of a code change. Now produce the structured adversarial review.
</role>

<task>
Based on your investigation in the prior turns of this thread, write up your findings as a structured review.
Target: {{TARGET_LABEL}}
User focus: {{USER_FOCUS}}
</task>

<finding_bar>
Report only material findings.
Do not include style feedback, naming feedback, low-value cleanup, or speculative concerns without evidence.
A finding should answer:
1. What can go wrong?
2. Why is this code path vulnerable?
3. What is the likely impact?
4. What concrete change would reduce the risk?
</finding_bar>

<structured_output_contract>
Return only valid JSON matching the provided schema.
Keep the output compact and specific.
Use `needs-attention` if there is any material risk worth blocking on.
Use `approve` only if you cannot support any substantive adversarial finding from your investigation.
Every finding must include:
- the affected file
- `line_start` and `line_end`
- a confidence score from 0 to 1
- a concrete recommendation
Write the summary like a terse ship/no-ship assessment, not a neutral recap.
</structured_output_contract>

<grounding_rules>
Be aggressive, but stay grounded.
Every finding must be defensible from what you read during the investigation.
Do not invent files, lines, code paths, incidents, attack chains, or runtime behavior you cannot support.
If a conclusion depends on an inference, state that explicitly in the finding body and keep the confidence honest.
</grounding_rules>

<calibration_rules>
Prefer one strong finding over several weak ones.
Do not dilute serious issues with filler.
If the change looks safe, say so directly and return no findings.
</calibration_rules>

<final_check>
Before finalizing, check that each finding is:
- adversarial rather than stylistic
- tied to a concrete code location
- plausible under a real failure scenario
- actionable for an engineer fixing the issue
</final_check>
48 changes: 48 additions & 0 deletions plugins/codex/prompts/adversarial-review-investigate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<role>
You are Codex performing an adversarial software review.
Your job is to break confidence in the change, not to validate it.
This is the investigation phase: gather evidence with read-only commands before producing any structured output.
</role>

<task>
Investigate the change so you can later produce a confident adversarial assessment.
Target: {{TARGET_LABEL}}
User focus: {{USER_FOCUS}}
</task>

<operating_stance>
Default to skepticism.
Assume the change can fail in subtle, high-cost, or user-visible ways until the evidence says otherwise.
Do not give credit for good intent, partial fixes, or likely follow-up work.
If something only works on the happy path, treat that as a real weakness.
</operating_stance>

<attack_surface>
Prioritize the kinds of failures that are expensive, dangerous, or hard to detect:
- auth, permissions, tenant isolation, and trust boundaries
- data loss, corruption, duplication, and irreversible state changes
- rollback safety, retries, partial failure, and idempotency gaps
- race conditions, ordering assumptions, stale state, and re-entrancy
- empty-state, null, timeout, and degraded dependency behavior
- version skew, schema drift, migration hazards, and compatibility regressions
- observability gaps that would hide failure or make recovery harder
</attack_surface>

<investigation_method>
Use read-only shell commands to inspect the diff and the surrounding code.
Useful starting points: `git diff`, `git log`, `git show`, `git blame`, `cat`, `rg`/`grep`.
Read the changed files, follow references, and confirm or refute hypotheses with evidence from the code.
Do not modify any files. Your sandbox is read-only.
{{REVIEW_COLLECTION_GUIDANCE}}
</investigation_method>

<convergence>
Continue investigating until you can defend a confident adversarial assessment.
When you have seen enough, emit a brief summary message describing what you found and stop running commands.
A summary message with no further command calls signals that you are ready for the finalization phase.
Do not produce a structured review yet — that comes in the next phase.
</convergence>

<repository_context>
{{REVIEW_INPUT}}
</repository_context>
69 changes: 59 additions & 10 deletions plugins/codex/scripts/codex-companion.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
interruptAppServerTurn,
parseStructuredOutput,
readOutputSchema,
runAppServerInvestigation,
runAppServerReview,
runAppServerTurn
} from "./lib/codex.mjs";
Expand Down Expand Up @@ -246,6 +247,24 @@ function buildAdversarialReviewPrompt(context, focusText) {
});
}

function buildAdversarialInvestigatePrompt(context, focusText) {
const template = loadPromptTemplate(ROOT_DIR, "adversarial-review-investigate");
return interpolateTemplate(template, {
TARGET_LABEL: context.target.label,
USER_FOCUS: focusText || "No extra focus provided.",
REVIEW_COLLECTION_GUIDANCE: context.collectionGuidance,
REVIEW_INPUT: context.content
});
}

function buildAdversarialFinalizePrompt(context, focusText) {
const template = loadPromptTemplate(ROOT_DIR, "adversarial-review-finalize");
return interpolateTemplate(template, {
TARGET_LABEL: context.target.label,
USER_FOCUS: focusText || "No extra focus provided."
});
}

function ensureCodexAvailable(cwd) {
const availability = getCodexAvailability(cwd);
if (!availability.available) {
Expand Down Expand Up @@ -404,14 +423,29 @@ async function executeReviewRun(request) {
}

const context = collectReviewContext(request.cwd, target);
const prompt = buildAdversarialReviewPrompt(context, focusText);
const result = await runAppServerTurn(context.repoRoot, {
prompt,
model: request.model,
sandbox: "read-only",
outputSchema: readOutputSchema(REVIEW_SCHEMA),
onProgress: request.onProgress
});
let result;
if (context.inputMode === "self-collect") {
const investigatePrompt = buildAdversarialInvestigatePrompt(context, focusText);
const finalizePrompt = buildAdversarialFinalizePrompt(context, focusText);
result = await runAppServerInvestigation(context.repoRoot, {
investigatePrompt,
finalizePrompt,
outputSchema: readOutputSchema(REVIEW_SCHEMA),
model: request.model,
sandbox: "read-only",
maxInvestigationTurns: request.maxInvestigationTurns,
onProgress: request.onProgress
});
} else {
const prompt = buildAdversarialReviewPrompt(context, focusText);
result = await runAppServerTurn(context.repoRoot, {
prompt,
model: request.model,
sandbox: "read-only",
outputSchema: readOutputSchema(REVIEW_SCHEMA),
onProgress: request.onProgress
});
}
const parsed = parseStructuredOutput(result.finalMessage, {
status: result.status,
failureMessage: result.error?.message ?? result.stderr
Expand All @@ -436,6 +470,9 @@ async function executeReviewRun(request) {
parseError: parsed.parseError,
reasoningSummary: result.reasoningSummary
};
if (result.investigation) {
payload.investigation = result.investigation;
}

return {
exitStatus: result.status,
Expand All @@ -445,7 +482,8 @@ async function executeReviewRun(request) {
rendered: renderReviewResult(parsed, {
reviewLabel: reviewName,
targetLabel: context.target.label,
reasoningSummary: result.reasoningSummary
reasoningSummary: result.reasoningSummary,
investigation: result.investigation ?? null
}),
summary: parsed.parsed?.summary ?? parsed.parseError ?? firstMeaningfulLine(result.finalMessage, `${reviewName} finished.`),
jobTitle: `Codex ${reviewName}`,
Expand Down Expand Up @@ -681,13 +719,23 @@ function enqueueBackgroundTask(cwd, job, request) {

async function handleReviewCommand(argv, config) {
const { options, positionals } = parseCommandInput(argv, {
valueOptions: ["base", "scope", "model", "cwd"],
valueOptions: ["base", "scope", "model", "cwd", "max-investigation-turns"],
booleanOptions: ["json", "background", "wait"],
aliasMap: {
m: "model"
}
});

const rawMaxTurns = options["max-investigation-turns"];
let maxInvestigationTurns;
if (rawMaxTurns !== undefined) {
const parsedTurns = Number.parseInt(rawMaxTurns, 10);
if (!Number.isFinite(parsedTurns) || parsedTurns <= 0) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-integer max investigation turn values

The new --max-investigation-turns validation uses Number.parseInt, which silently accepts malformed values like 1.5 (becomes 1) or 10abc (becomes 10) even though the flag contract says this must be a positive integer. This can hide user typos and run significantly fewer or different investigation turns than requested, degrading review depth without any error signal. Please validate the full token (for example with a strict integer regex or Number(...) plus an integer check) before accepting it.

Useful? React with 👍 / 👎.

throw new Error(`--max-investigation-turns must be a positive integer (got: ${rawMaxTurns})`);
}
maxInvestigationTurns = parsedTurns;
}

const cwd = resolveCommandCwd(options);
const workspaceRoot = resolveCommandWorkspace(options);
const focusText = positionals.join(" ").trim();
Expand Down Expand Up @@ -716,6 +764,7 @@ async function handleReviewCommand(argv, config) {
model: options.model,
focusText,
reviewName: config.reviewName,
maxInvestigationTurns,
onProgress: progress
}),
{ json: options.json }
Expand Down
Loading