Add Skills capability for progressive tool loading

[DouweM]

Summary

• Implements a Skills capability (AbstractCapability subclass) that enables progressive tool loading: agents discover skills via search_skills(query) and activate them via load_skill(name), keeping unloaded tools hidden from the model's context window
• Skills can be defined in Python (with callable tools or FunctionToolset) or loaded from markdown files with YAML frontmatter (pure knowledge packages)
• Per-run state isolation via for_run(), spec-serializable via from_spec(dirs=[...]), and tool visibility controlled through the prepare_tools hook

Closes #22. Partially addresses #40.

Test plan

• 47 new tests covering all code paths (48 total with existing placeholder test)
• Skill dataclass: name validation, tool name extraction, defaults
• Markdown parsing: frontmatter extraction, error cases, multi-line bodies
• Directory loading: .md discovery, non-md filtering, empty dirs
• Skills capability: instructions, toolset assembly, serialization
• Meta-tools: search matching/case-insensitivity, load success/failure, loaded status
• prepare_tools: hides unloaded skill tools, shows loaded, preserves non-skill tools
• for_run: state isolation between runs
• from_spec: directory-based construction
• All passing: ruff lint, ruff format, pyright strict, pytest

🤖 Generated with Claude Code

[devin-ai-integration]

Devin Review found 2 potential issues.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🟡 Frontmatter closing delimiter search matches --- inside values, not just on its own line

_parse_skill_markdown at src/pydantic_harness/skills.py:117 uses stripped.find('---', 3) to locate the closing frontmatter delimiter. This performs a simple substring search, so it will match --- appearing within a frontmatter value (e.g., description: A---B) rather than requiring --- to be on its own line, which is the standard YAML frontmatter convention. When triggered, the frontmatter is truncated at the embedded ---, the description (or other value) is silently cut short, and the remainder is incorrectly treated as the body/instructions.

Example of incorrect parse

Input:

---
name: my-skill
description: Long---description
---
Body text

find('---', 3) matches the --- inside Long---description at character 34 instead of the actual closing delimiter. Result: description is parsed as Long, and body becomes description\n---\nBody text.

Suggested change

	end = stripped.find('---', 3)
	end = stripped.find('\n---', 3)
	if end == -1:
	raise ValueError(f'Unclosed YAML frontmatter in {source}')
	frontmatter_text = stripped[3:end].strip()
	body = stripped[end + 4 :].strip() or None

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 for_run clone shares skills list reference — meta-tool binding depends on get_ re-extraction*

The for_run method at src/pydantic_harness/skills.py:200 creates a clone via Skills(skills=self.skills), sharing the same skills list by reference. The critical design question is whether get_toolset() is re-called on the clone after for_run. The AbstractCapability.for_run docstring says it is "Called once per run, before get_*() re-extraction", which implies get_toolset() is indeed re-invoked on the clone. If so, the meta-tools (_search_skills, _load_skill, _unload_skill) would be bound methods of the clone, and _loaded_skill_names mutations during a run would correctly affect the same instance that prepare_tools checks. This is correct under the documented lifecycle. However, if any future pydantic-ai version changes the re-extraction behavior, this would silently break — the meta-tools would mutate the original instance's state while prepare_tools reads from the clone's state, making skill loading appear to have no effect.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[DouweM]

Originally posted by @DouweM in #133 comment (PR was recreated)

Audit vs prior art: Skills

Worth adding now:

• agentskills.io tolerance: ignore unknown frontmatter keys instead of failing
• unload_skill(name) tool to free context window
• Fuzzy/word-boundary search in search_skills

Follow-up opportunities:

• Remote skill registries (git-based, like VStorm)
• Dependency resolution between skills
• Resource/script separation (agentskills.io pattern)

[mtessar]

@DouweM i am excited to see skills being added. Thanks for working on it!

[dergachoff]

Do I understand correctly that dynamic loading/unloading skills breaks cache?

[DouweM]

It's clear that the people want skills support :)

From discussing on Slack (join, thread), it's also clear that people mean different things when they say that: some are you really looking for full filesys/shell user-provided skills support, but many primarily want "programmatic skills" that are defined code-side (and could be loaded from a file or a DB, but not necessarily the user's own/sandbox FS).

Check out the thread Slack thread if you have opinions. We're meeting next week with a couple of champions from the Slack thread to make sure we build the right thing, and not get distracted by agentskills.io if we don't have to.