Skip to content

Commit daa1e70

Browse files
raiderrobertraiderrobert
authored
feat: add codebase-audit skill (#37)
* feat: add codebase-audit skill Structural review skill that evaluates codebase organization against a standard checklist (boundaries, cohesion, error handling, code hygiene, documentation alignment) plus project-specific conventions derived from architecture docs. * fix: add codebase-audit to README catalog and add common mistakes section - Add codebase-audit skill entry to README Skills Catalog (fixes CI validate-readme) - Add Common Mistakes section to SKILL.md with actionable audit pitfalls
1 parent 1c6556a commit daa1e70

3 files changed

Lines changed: 353 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Skills that keep you in the flow. Domain expertise baked in so you don't have to
66

77
| Skill | Description |
88
|-------|-------------|
9+
| [codebase-audit](skills/codebase-audit/SKILL.md) | Structural review of codebase organization against a standard checklist plus project conventions |
910
| [design-audit](skills/design-audit/SKILL.md) | Post-generation review catching AI design defaults |
1011
| [design-frontend](skills/design-frontend/SKILL.md) | Pre-set aesthetic constraints for web components |
1112
| [design-preflight](skills/design-preflight/SKILL.md) | Design constraints gate before Pencil .pen file work |

‎docs/specs/2026-04-22-codebase-audit-design.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Codebase Audit Skill — Design Spec
2+
3+
**Goal:** A skill that performs a comprehensive structural review of a codebase's organization, looking for architectural smells and quality issues. Not a bug hunt or PR review — an audit of how code is organized.
4+
5+
**Skill name:** `codebase-audit`
6+
**Location:** `skills/codebase-audit/SKILL.md`
7+
**Repo:** `flow`
8+
9+
---
10+
11+
## Trigger Phrases
12+
13+
codebase audit, structural review, architecture audit, code organization review, audit this codebase, how is this repo organized, code health check, audit the code
14+
15+
Distinct from existing skills:
16+
- `review-and-fix` — PR-scoped, includes fix dispatch
17+
- `iterative-review-fix` — repeated review-fix loop on a branch
18+
- `review-with-research` — validates approach soundness with external research
19+
- `design-audit` — visual/frontend design quality
20+
21+
---
22+
23+
## Workflow
24+
25+
### 1. Read project docs
26+
27+
Read AGENTS.md, ARCHITECTURE.md, CLAUDE.md, CONTRIBUTING.md, README.md (whatever exists). AGENTS.md is a good starting place as it often documents project conventions in agent-friendly form. Extract the project's own documented conventions and boundaries. These become the "project-specific" criteria.
28+
29+
### 2. Map the codebase
30+
31+
Get the shape before reading individual files:
32+
- File tree and directory structure
33+
- Line counts per module/file
34+
- Import/dependency graph (which modules import which)
35+
36+
This step identifies the largest files, the most-imported modules, and the overall layering.
37+
38+
### 3. Read source files
39+
40+
For codebases under ~10K lines, read all source files. For larger codebases, sample strategically:
41+
- Public API surfaces (exports, interfaces, `__init__` / index files)
42+
- Module boundaries (top-level of each package/directory)
43+
- Largest files (most likely to have cohesion issues)
44+
- Files with the most imports or most importers (coupling hotspots)
45+
46+
### 4. Evaluate against checklist
47+
48+
Apply the default checklist plus project-specific criteria derived from step 1.
49+
50+
### 5. Produce findings
51+
52+
Structured report organized by file, with severity, criterion, description, and recommendation.
53+
54+
---
55+
56+
## Default Checklist
57+
58+
Six categories, grounded in established software engineering principles (Fowler's code smells, coupling/cohesion metrics, architecture review literature).
59+
60+
### Boundaries & Dependencies
61+
62+
- **Dependency direction violations** — does the documented or implied layering hold? Do lower layers import from higher layers?
63+
- **Circular dependencies** — deferred/lazy imports used to break import cycles, or mutual function-level coupling between modules
64+
- **Private API access** — modules reaching into another module's private internals (underscore-prefixed in Python, unexported in Go, `pub(crate)` in Rust, non-exported in TS)
65+
- **Leaky abstractions** — implementation details visible in public interfaces
66+
- **Feature envy** — code that uses another module's data or functions more than its own
67+
- **Fan-out hotspots** — modules with too many dependencies (high coupling)
68+
- **Hidden dependencies** — dependencies not visible from the interface (global state, implicit ordering)
69+
70+
### Cohesion & Responsibility
71+
72+
- **God files/modules** — too large, too many responsibilities, too many reasons to change
73+
- **Divergent change** — one module that changes for many unrelated reasons (low cohesion)
74+
- **Shotgun surgery** — one logical change requires edits scattered across many files
75+
- **Speculative generality** — unused abstractions, interfaces nobody implements, parameters nobody passes (YAGNI violations)
76+
- **Data clumps** — groups of values that always travel together but aren't a named type
77+
78+
### Error Handling
79+
80+
- **Overly broad error handling** — catching all errors instead of specific types (`except Exception` in Python, untyped `catch(e)` in TS, `recover()` swallowing panics in Go, `.unwrap()` throughout in Rust)
81+
- **Inconsistent error strategies** — similar modules handling errors in fundamentally different ways
82+
- **Wrong-layer error handling** — errors caught too early (swallowed before the caller can act) or too late (propagated without context)
83+
- **Unused error types** — custom error types defined but not raised/returned
84+
85+
### Code Hygiene
86+
87+
- **Duplicated logic** — repeated code across modules that should be extracted
88+
- **Dead code** — unused functions, unreachable branches, commented-out code
89+
- **Magic strings/numbers** — hardcoded values that should be named constants
90+
- **Inconsistent patterns** — similar operations done differently in different places for no good reason
91+
92+
### Documentation Alignment
93+
94+
- **Stale architecture docs** — documentation that doesn't match the actual code
95+
- **Documented but unenforced conventions** — rules written down that the code violates
96+
- **Undocumented conventions** — implicit rules the code follows that aren't written anywhere
97+
98+
### Project-Specific
99+
100+
- Whatever conventions the project documents in AGENTS.md, ARCHITECTURE.md, CLAUDE.md, or CONTRIBUTING.md
101+
- Examples: "core/ must not import from commands/", "all commands export `run_<command>()`", "type hints on all signatures"
102+
103+
---
104+
105+
## Finding Format
106+
107+
```
108+
### [file_path:line]
109+
110+
**Criterion:** [checklist item name]
111+
**Severity:** HIGH | MEDIUM | LOW
112+
**Description:** [what's wrong, with code snippet]
113+
**Recommendation:** [how to fix]
114+
```
115+
116+
Severity definitions:
117+
- **HIGH** — architectural violation, likely bug source, or data loss risk
118+
- **MEDIUM** — code smell that meaningfully hurts maintainability
119+
- **LOW** — style/convention nit or minor inconsistency
120+
121+
Findings organized by file, not by criterion — makes them actionable.
122+
123+
---
124+
125+
## Report Structure
126+
127+
```
128+
# Codebase Audit: [project name]
129+
130+
**Date:** YYYY-MM-DD
131+
**Scope:** [files reviewed, line count]
132+
133+
## Summary
134+
[2-3 sentences: finding count, severity breakdown, biggest themes]
135+
136+
## Findings
137+
[organized by file, using the finding format above]
138+
139+
## Themes
140+
[recurring patterns across findings — what systemic issues do they point to?]
141+
142+
## What's Working Well
143+
[positive observations — validated conventions, clean boundaries, good patterns]
144+
```
145+
146+
---
147+
148+
## What This Skill Does NOT Do
149+
150+
- **Bug hunting** — this is structural, not functional
151+
- **Security audit** — no OWASP/CVE scanning
152+
- **Performance profiling** — no hot path analysis
153+
- **PR review** — use `review-and-fix` for that
154+
- **Fix dispatch** — this skill produces a report, not fixes. The user decides what to act on.
155+
- **Automated fixes** — pair with `review-and-fix` or `iterative-review-fix` if you want automated remediation
156+
157+
---
158+
159+
## Composability
160+
161+
This skill is a single-agent, single-pass audit. For more rigor:
162+
- Wrap with `diverge-critique-converge` for multi-perspective review
163+
- Follow up with `review-and-fix` or `iterative-review-fix` to act on findings
164+
- Use findings to create GitHub issues via `gh issue create`

‎skills/codebase-audit/SKILL.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
---
2+
name: codebase-audit
3+
description: "Use when reviewing a codebase's structural organization, not a PR or branch. Triggers on: codebase audit, structural review, architecture audit, code organization review, audit this codebase, how is this repo organized, code health check, audit the code. Produces a findings report against a standard checklist plus project-specific conventions."
4+
---
5+
6+
# Codebase Audit
7+
8+
Comprehensive structural review of how a codebase is organized. Evaluates architectural boundaries, code smells, error handling patterns, and documentation alignment against a standard checklist plus the project's own documented conventions.
9+
10+
This is not a bug hunt, security audit, or PR review. It examines organization and structure.
11+
12+
---
13+
14+
## Workflow
15+
16+
### 1. Read project docs
17+
18+
Read whatever exists: AGENTS.md, ARCHITECTURE.md, CLAUDE.md, CONTRIBUTING.md, README.md. AGENTS.md is a good starting place — it often documents conventions in agent-friendly form.
19+
20+
Extract two things:
21+
- **Documented boundaries** — layer rules, import restrictions, module responsibilities
22+
- **Documented conventions** — naming, patterns, error handling, testing requirements
23+
24+
These become the project-specific checklist items in step 4.
25+
26+
### 2. Map the codebase
27+
28+
Get the shape before reading individual files:
29+
30+
Use language-appropriate tools:
31+
- File tree and line counts (`find`, `wc -l`, or language-specific tools)
32+
- Import/dependency graph (grep for import statements, or use `cargo tree`, `go mod graph`, etc.)
33+
34+
Identify:
35+
- Largest files (cohesion risk)
36+
- Most-imported modules (coupling hotspots, fan-in)
37+
- Modules with the most imports (fan-out hotspots)
38+
- Overall layering and directory structure
39+
40+
### 3. Read source files
41+
42+
**Under ~10K lines:** Read all source files.
43+
44+
**Over ~10K lines:** Sample strategically:
45+
- Public API surfaces (exports, package-level files, index/init modules)
46+
- Module boundaries (top-level of each package/directory)
47+
- Largest files (most likely to have cohesion problems)
48+
- Files with the most imports or the most importers
49+
50+
### 4. Evaluate against checklist
51+
52+
Apply the default checklist below, plus project-specific criteria from step 1.
53+
54+
For each potential finding:
55+
- Verify it against the actual code (cite file and line)
56+
- Assign severity
57+
- Write a concrete recommendation
58+
59+
Skip anything that is technically true but does not matter in practice.
60+
61+
### 5. Produce report
62+
63+
Write the report using the output format below. Organize findings by file, not by criterion.
64+
65+
---
66+
67+
## Default Checklist
68+
69+
### Boundaries & Dependencies
70+
71+
| Smell | What to look for |
72+
|-------|-----------------|
73+
| Dependency direction | Lower layers importing from higher layers, violating documented or implied architecture |
74+
| Circular dependencies | Deferred/lazy imports to break cycles, mutual function-level coupling between modules |
75+
| Private API access | Modules reaching into another module's internals (language-specific visibility rules) |
76+
| Leaky abstractions | Implementation details visible in public interfaces |
77+
| Feature envy | Code that uses another module's data/functions more than its own |
78+
| Fan-out hotspots | Modules with too many dependencies |
79+
| Hidden dependencies | Dependencies not visible from the interface — global state, implicit ordering, side-effect coupling |
80+
81+
### Cohesion & Responsibility
82+
83+
| Smell | What to look for |
84+
|-------|-----------------|
85+
| God files | Too large, too many responsibilities, too many reasons to change |
86+
| Divergent change | One module that changes for many unrelated reasons |
87+
| Shotgun surgery | One logical change requires edits scattered across many files |
88+
| Speculative generality | Unused abstractions, interfaces nobody implements, parameters nobody passes |
89+
| Data clumps | Groups of values that always travel together but aren't a named type |
90+
91+
### Error Handling
92+
93+
| Smell | What to look for |
94+
|-------|-----------------|
95+
| Overly broad handling | Catching all errors instead of specific types |
96+
| Inconsistent strategies | Similar modules handling errors in fundamentally different ways |
97+
| Wrong-layer handling | Errors swallowed before the caller can act, or propagated without context |
98+
| Unused error types | Custom error types defined but never raised or returned |
99+
100+
### Code Hygiene
101+
102+
| Smell | What to look for |
103+
|-------|-----------------|
104+
| Duplicated logic | Repeated code across modules that should be extracted |
105+
| Dead code | Unused functions, unreachable branches, commented-out code |
106+
| Magic values | Hardcoded strings or numbers that should be named constants |
107+
| Inconsistent patterns | Similar operations done differently in different places for no reason |
108+
109+
### Documentation Alignment
110+
111+
| Smell | What to look for |
112+
|-------|-----------------|
113+
| Stale docs | Architecture documentation that doesn't match the actual code |
114+
| Unenforced conventions | Rules written down that the code violates |
115+
| Undocumented conventions | Implicit rules the code follows that aren't written anywhere |
116+
117+
### Project-Specific
118+
119+
Derived from step 1. Whatever conventions the project documents as its own rules.
120+
121+
Examples: "core/ must not import from commands/", "all handlers return a Result type", "every module has a corresponding test file"
122+
123+
---
124+
125+
## Output Format
126+
127+
```markdown
128+
# Codebase Audit: [project name]
129+
130+
**Date:** YYYY-MM-DD
131+
**Scope:** [files reviewed, total line count]
132+
133+
## Summary
134+
135+
[2-3 sentences: finding count by severity, biggest themes]
136+
137+
## Findings
138+
139+
### src/path/to/file.ext:42
140+
141+
**Criterion:** [checklist item name]
142+
**Severity:** HIGH | MEDIUM | LOW
143+
**Description:** [what's wrong, with code snippet]
144+
**Recommendation:** [how to fix]
145+
146+
### src/path/to/other.ext:17
147+
148+
...
149+
150+
## Themes
151+
152+
[Recurring patterns across findings — what systemic issues do they point to?]
153+
154+
## What's Working Well
155+
156+
[Positive observations — validated conventions, clean boundaries, good patterns worth preserving]
157+
```
158+
159+
**Severity definitions:**
160+
- **HIGH** — architectural violation, likely bug source, or data loss risk
161+
- **MEDIUM** — code smell that meaningfully hurts maintainability
162+
- **LOW** — style or convention nit
163+
164+
---
165+
166+
## Common Mistakes
167+
168+
| Mistake | Fix |
169+
|---------|-----|
170+
| Reporting every code smell regardless of impact | Skip findings that are technically true but don't matter in practice |
171+
| Auditing PR diff instead of full codebase | This skill evaluates structure, not changes. Use `review-and-fix` for diffs |
172+
| Omitting project-specific conventions from AGENTS.md/CLAUDE.md | Always read project docs first — project-specific criteria often surface the most actionable findings |
173+
| Organizing findings by criterion instead of by file | Findings grouped by file are actionable; by criterion is academic |
174+
| Treating HIGH severity as "fix immediately" | HIGH means architectural risk or likely bug source, not urgency. Let the user prioritize |
175+
176+
## Scope Boundaries
177+
178+
This skill produces a report. It does not fix anything.
179+
180+
| Need | Use instead |
181+
|------|-------------|
182+
| Review a PR or branch diff | `review-and-fix` |
183+
| Repeated review-fix cycles | `iterative-review-fix` |
184+
| Validate whether an approach is sound | `review-with-research` |
185+
| Audit visual/frontend design | `design-audit` |
186+
| Multi-perspective review | Wrap this skill with `diverge-critique-converge` |
187+
| Automated remediation | Follow up findings with `review-and-fix` or `iterative-review-fix` |
188+
| Track findings as tickets | Use `gh issue create` on each finding |

0 commit comments

Comments
 (0)