feat(claude-agent-sdk): opt-in setting_sources for target-repo skills

[João Mena (joaomena)]

Fixes #501.

Problem

ClaudeAgentSdkProvider passed setting_sources=[] unconditionally, so no workflow could load the settings tier of the repository it operates on. An agent working a target repo could not use that repo's own .claude/skills/, CLAUDE.md, or .claude/rules/*.md — those conventions had to be duplicated into the workflow's prompts.

What changed

Adds an opt-in runtime.provider.setting_sources (user / project / local), defaulting to [] so behaviour is unchanged unless a workflow asks for it.

The empty default is load-bearing rather than cosmetic: the SDK re-defaults an unset setting_sources to ["user", "project"] whenever skills is set, so [] has to be sent explicitly to keep a run hermetic. That property is pinned by a test.

Two things this needed beyond the plumbing, both found by running it rather than by reading:

1. Grant the Skill tool when setting_sources is non-empty. CLI-discovered skills never pass through skill_names, so gating on that alone listed a repo's skills to the model with no tool to invoke them — discovery without execution.
2. Resolve ClaudeAgentOptions.skills to "all" when tiers are enabled and the workflow named no skills. Otherwise every call failed with "not in this session's skills allowlist" — observed with 28 discovered skills, all rejected. A declared skills:/plugins: list still wins; discovery does not widen what the author asked for.

Security note

project loads the entire tier, hooks included — pointing it at untrusted code runs that code's hooks. Documented on the schema field. The [] default means nobody gets this without asking for it.

Whether user should be permitted at all is a fair question to settle in review: it makes a run depend on the operator's machine, which is why it is available but not something we use.

Verification

Tested against claude-agent-sdk 0.2.87 with a fixture repo whose .claude/rules is a symlink to a tool-agnostic .agents/rules, and a CLAUDE.md that never references it:

• setting_sources=[] → the agent reports no rule exists
• setting_sources=["project"] → the agent returns the rule's contents and its marker token

So skills, CLAUDE.md, and .claude/rules all arrive through the one tier, symlinks included.

New tests were checked by mutation: reverting the Skill-tool gate fails test_declared_sources_grant_the_skill_tool, and reverting the filter fails the two "all" tests.

Note for reviewers

tests/test_providers/ and tests/test_config/ pass in full (2683 passed, 1 skipped).

Three tests fail on my machine both with and without this change, so they look environment-dependent rather than related:

• tests/test_skills/test_path_entries.py::TestUnreadableParent::test_unreadable_parent_is_reported_not_raised_raw
• tests/test_skills/test_path_entries.py::TestSkillsRootDiagnostics::test_mis_cased_skill_md_is_reported
• tests/test_plugins/test_registry.py::TestUnreadableTrees::test_unreadable_skill_subdirectory_is_reported

All three depend on a chmod-unreadable path, which does not take effect for my user. Confirmed by running them on a clean checkout of main.

[João Mena (joaomena)]

João Mena (João Mena (@joaomena)) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Too Good To Go"

[Jason Robert (jrob5756)]

Five blocking findings here, all tied to the same root cause: setting_sources was added to the schema and to the claude-agent-sdk provider without the enforcement, serialization, and per-agent scoping that every other structured provider field already has. None of them are style nits — b1/b2 mean the field can be silently accepted and then silently dropped, and b3/b4 mean the skill filtering it interacts with doesn't do what its own comments claim. Until those are fixed this shouldn't merge as-is.

Blocking findings, by location:

• src/conductor/config/schema.py:2693 — setting_sources is documented claude-agent-sdk-only but accepted (and silently ignored) on every other provider.
• src/conductor/config/schema.py:2692 — has_structured_config() doesn't know about the field, so it's erased by serialization and dropped by --provider overrides with no warning.
• src/conductor/providers/claude_agent_sdk.py:1016 — a per-agent skills: [] opt-out gets silently upgraded to skills="all" once a settings tier is enabled.
• src/conductor/providers/claude_agent_sdk.py:1012 — the comment justifying the "all" branch describes SDK behavior that isn't what the SDK actually does, and it contradicts an unchanged comment nine lines above it.

One more blocking finding has no single line to anchor to, so it's here in full:

BLOCKING — the invariant this PR overturns is still documented as unconditional in AGENTS.md and two user-facing docs, and there's no CHANGELOG entry.

The diff touches four files and no documentation. These statements are now false:

Location	Now-false text
AGENTS.md:434	"setting_sources=[] unconditionally, for the same reason strict_mcp_config=True is unconditional"
AGENTS.md:431	"the unconditional setting_sources=[] ... stops the CLI loading CLAUDE.md, project settings, and hooks from it"
AGENTS.md:217	"keeps enable_config_discovery off on Copilot ... and setting_sources=[] on claude-agent-sdk"
AGENTS.md:435	"the SDK auto-allows it via Skill(<name>) in allowed_tools" — on the new "all" path there's no <name>; the SDK appends the bare Skill, a broader auto-approve
docs/providers/comparison.md:164	"Conductor also pins the SDK's setting_sources to an empty list on every run", followed by a flat list of things not inherited (skills, CLAUDE.md, settings.json, hooks)
docs/providers/experimental.md:104	"setting_sources is pinned empty as of #352 so ambient instructions, settings, hooks, and skills are not inherited"
src/conductor/skills/discovery.py:20	still asserts setting_sources=[] as an unconditional invariant

AGENTS.md is loaded as agent instructions in this repo, so a stale invariant there won't just sit unread — it'll get asserted as fact to whoever touches this file next, human or agent. comparison.md and experimental.md are worse for users specifically, since both currently promise an unconditional safety guarantee about hooks not being inherited, and this PR makes that guarantee conditional without saying so. The field's own docstring already warns to enable it only for repos trusted as much as the workflow itself — the docs should carry that same caveat, not the stronger claim.

Also missing: the ## [Unreleased] section of CHANGELOG.md is empty, docs/configuration.md's field-compatibility table has no row for this field, docs/workflow-syntax.md never mentions the new key, and no example workflow exercises it (so make validate-examples never touches this path).

Suggested fix: rewrite the AGENTS.md, comparison.md, experimental.md, and skills/discovery.py statements as "empty by default, opt-in per workflow via runtime.provider.setting_sources" and carry the trust caveat forward with them. Add the CHANGELOG entry, the workflow-syntax doc, the compatibility-table row, and an example.

Findings that could not be anchored inline

These name a line outside this pull request's diff, so GitHub cannot attach them to a specific line.

tests/test_providers/test_claude_agent_sdk.py:1

RECOMMENDED

The ten new tests cover provider wiring on the happy path and the _resolve_skill_filter table well. What's missing maps one-to-one onto the blocking findings above — each would have been caught by a three-line test:

• No schema test. grep -rn setting_sources tests/ matches one file. Nothing asserts the field is rejected on a non-claude-agent-sdk provider, nothing rejects an invalid tier string, nothing round-trips model_dump/model_validate.
• No argv assertions, even though TestSkillsWiring's own docstring sets the standard for this subsystem: "The provider's contract is ultimately the claude CLI command line, so these assert the argv the SDK builds from our options rather than stopping at the options object." That matters here specifically because the "all" branch produces a structurally different --allowedTools value (bare Skill) from the declared branch (Skill(name)), and on the tools: [] path permission_mode is None, so --allowedTools is the only thing granting the tool. If the bare-Skill injection regressed, test_declared_sources_grant_the_skill_tool would still pass while every skill call got refused — the exact failure this PR says it exists to prevent.
• The combined path never runs through execute. setting_sources plus workflow-declared skills is only covered at the pure-function level; a refactor that let setting_sources clobber the declared allowlist would leave the unit test green.
• Only the project tier is exercised. user, local, and multi-element lists (which comma-join into argv) are untested.
• tools: omitted + setting_sources (the claude_code preset path) is untested.

Three of the new tests are also strictly weaker duplicates of pre-existing argv-level ones: test_default_sends_an_explicit_empty_list vs test_setting_sources_isolated_unconditionally, test_no_sources_no_skills_withholds_the_skill_tool vs test_explicit_no_tools_without_skills_stays_empty, and test_default_still_enables_no_skills vs test_no_skills_suppresses_ambient_discovery.

Separately, test_setting_sources_isolated_unconditionally (line 2884, not modified by this PR) is now misnamed — its docstring reads "No ambient skills, CLAUDE.md, settings.json, or hooks — ever." It still passes, but the name and "ever" assert a guarantee the code no longer makes, and a future reader will trust it when reasoning about the security boundary.

Suggested fix: add a parametrized schema-rejection test for non-claude-agent-sdk providers, a model_dump round-trip test, an argv assertion that the "all" path emits bare Skill in --allowedTools, a skills: [] + setting_sources test, and parametrize the tier over [["project"], ["user"], ["local"], ["user", "project", "local"]]. Reuse TestSkillsWiring._capture_options/_argv (add a setting_sources= parameter) instead of hand-rolling fake_query per test, and rename test_setting_sources_isolated_unconditionally to ..._by_default.

[Jason Robert (jrob5756)]

BLOCKING

The docstring says "claude-agent-sdk-only," but _check_field_compatibility (schema.py:2850-2919) never enforces it. Every other provider-scoped field on this model does — the seven copilot_only_fields, auth_token via claude_only_fields, the seven aca_only_fields, four individual hermes_* guards. setting_sources is in none of them.

Confirmed empirically, all of these construct without error:

ProviderSettings(name='copilot',  setting_sources=['project'])  -> ACCEPTED
ProviderSettings(name='openai',   setting_sources=['project'])  -> ACCEPTED
ProviderSettings(name='claude',   setting_sources=['user'])     -> ACCEPTED
ProviderSettings(name='hermes',   setting_sources=['local'])    -> ACCEPTED
ProviderSettings(name='aca', ..., setting_sources=['project'])  -> ACCEPTED

Only the case "claude-agent-sdk" arm at factory.py:264 reads the field, so on every other provider this is a pure no-op. aca is the sharpest case: set inner_provider: claude-agent-sdk alongside setting_sources: [project], the schema accepts it, and the runner's four-key inner_provider_settings allowlist means it's never forwarded to the sandbox.

This one matters more than a typical no-op because the field is security-relevant — the user is explicitly asking to load ambient hooks — and there's no feedback that the request was dropped. It also breaks the promise in docs/configuration.md:236: "The schema rejects the following misconfigurations at config load time so they cannot silently produce a no-op SDK call."

conductor validate doesn't cover this either: validate_workflow_config is only imported by cli/validate.py, and conductor run never calls it. The model_validator is the only gate on the run path.

Suggested change

	"""Claude Code settings tiers the session may load. claude-agent-sdk-only.
	@model_validator(mode="after")
	def _check_field_compatibility(self) -> "ProviderSettings":
	if self.setting_sources is not None and self.name != "claude-agent-sdk":
	raise ValueError(
	"'setting_sources' is only supported when name='claude-agent-sdk' "
	f"(got name={self.name!r}). It selects Claude Code settings tiers, "
	"which no other provider reads."
	)

Match this to the existing hermes_home shape at schema.py:2909, and add a parametrized rejection test alongside the existing per-provider scoping tests.

[Jason Robert (jrob5756)]

BLOCKING

has_structured_config() (schema.py:3085) is has_custom_routing() or has_external_runtime() or has_aca_config(). None of the three looks at setting_sources, so a settings object that sets only this field returns False. The @model_serializer at schema.py:3089 then collapses it to a bare string:

p = ProviderSettings(name='claude-agent-sdk', setting_sources=['project'])
p.has_structured_config()  # False
p.model_dump()             # 'claude-agent-sdk'   <- the opt-in is gone
ProviderSettings.model_validate(p.model_dump()).setting_sources  # None

AGENTS.md states the contract this breaks directly: "has_structured_config() keeps either mode from collapsing to bare-string serialization." This PR adds a third mode and doesn't extend the guard.

Two consequences are already live in the tree:

1. cli/run.py:241 — _apply_provider_override only warns "Provider override discards structured runtime.provider settings" when had_structured is true. --provider claude-agent-sdk (or conductor resume --provider ..., which AGENTS.md documents as routine) against a workflow with setting_sources: [project] discards the setting at line 249 with no warning — every other structured field gets one.
2. cli/run.py:206 — _describe_provider short-circuits on the same predicate and returns the bare name, so even -v never shows that ambient settings tiers are active. For a toggle that enables arbitrary hook execution, that's the wrong thing to stay quiet about.

Resume happens to be safe today only because engine/checkpoint.py doesn't persist runtime.provider and resume re-reads the YAML — that's luck, not design.

Suggested change

	setting_sources: list[Literal["user", "project", "local"]] | None = None
	def has_structured_config(self) -> bool:
	"""Return True when the provider has any non-default structured settings."""
	return (
	self.has_custom_routing()
	or self.has_external_runtime()
	or self.has_aca_config()
	or self.setting_sources is not None
	)

Also add a setting_sources=[...] part to cli/run.py::_describe_provider, plus a round-trip regression test: ProviderSettings.model_validate(p.model_dump()) == p.

[Jason Robert (jrob5756)]

BLOCKING

skill_names comes from _resolve_skill_plugins(skill_directories) at line 904. An agent that declares skills: [] — the documented explicit opt-out — resolves to no skill directories, so skill_names == []. With workflow-level setting_sources: [project], _resolve_skill_filter([], ['project']) returns "all", so the agent that explicitly asked for no skills gets every skill the target repo's .claude/skills ships.

Line 945 makes it worse: skills_enabled=bool(skill_names) or bool(self._setting_sources) grants the Skill tool back to an agent that declared both tools: [] and skills: [].

This contradicts two documented invariants — AGENTS.md's "skills: [] remains the one opt-out," and docs/providers/comparison.md:164, which describes pinning setting_sources empty as exactly what stopped skills: [] from being a no-op on this provider. This PR reintroduces that no-op at the per-agent level.

The asymmetry is structural: setting_sources lives on the workflow-global runtime.provider, while skills: is per-agent, so there's currently no way to say "this workflow loads the target repo's skills, but this one agent gets none."

Suggested change

	skills=_resolve_skill_filter(skill_names, self._setting_sources),
	effective_sources = [] if agent.skills == [] else self._setting_sources
	sdk_tools, permission_mode = self._resolve_tool_config(
	tools,
	agent,
	skills_enabled=bool(skill_names) or bool(effective_sources),
	agents_enabled=bool(custom_agents),
	)

_resolve_tool_config already reads the raw tri-state (agent.tools is None) for exactly this reason — do the same with agent.skills, and pass effective_sources into _resolve_skill_filter(skill_names, effective_sources) as well. Add a test for skills: [] + setting_sources: [project].

[Jason Robert (jrob5756)]

BLOCKING

The comment claims that with skills=[] "the model lists the repo's skills (the listing leaks past this filter) and every invocation comes back 'not in this session's skills allowlist.'" That's not what the SDK does.

The installed SDK's own docstring for ClaudeAgentOptions.skills says the opposite: it's "a context filter, not a sandbox — unlisted skills are hidden from the model's listing and rejected by the Skill tool, but their files remain on disk." The bundled CLI backs this up: the function building the skill_listing attachment filters by the session skill allowlist and returns [] when nothing survives, and returns [] outright when the session holds no Skill tool. The "not in this session's skills allowlist" string is real, but it's the invocation backstop, reachable only if the model names a skill it was never shown. The premise that the model sees the skill anyway is wrong.

This isn't just a wording problem:

1. It contradicts the unchanged comment at line 1003 (correct, pre-existing): "unlisted skills are hidden from the model's listing and rejected by the Skill tool." A reader can't tell which comment to trust, and the wrong one is newer.
2. The real behavior argues for a narrower fix. If skills=[] hides discovered skills rather than causing a rejection loop, the actual problem is "the tier loads them and then hides all of them" — and "all" is a blunt answer to that. Opting into ["user"] now enables every skill in ~/.claude/skills, which deserves to be a deliberate, stated tradeoff rather than a side effect of a mis-described failure mode.

The same false premise is repeated at lines 297-303 (_resolve_skill_filter docstring), 940-944 (skills_enabled comment), 1543-1545 (_resolve_tool_config docstring), and in the test docstring at tests/test_providers/test_claude_agent_sdk.py:2357-2360 — five copies to fix.

Suggested change

	# (the listing leaks past this filter) and every invocation comes
	# Skills discovered from a settings tier never pass through `skill_names`.
	# Sending `[]` sets the CLI's session skill allowlist to empty, which
	# suppresses them from the model's listing entirely -- so enabling a tier
	# would load the repo's skills and then hide every one of them. `"all"`
	# omits the filter (the SDK treats `"all"` and omitted as equivalent at the
	# wire level), leaving exactly what the enabled tiers discovered.

And confirm that widening to "all" — rather than, say, refusing the combination — is actually the intended tradeoff for the ["user"] tier.

[Jason Robert (jrob5756)]

RECOMMENDED

provider_settings is typed ProviderSettings | None and can't be a bare string here (a legacy string is normalized to provider_type with provider_settings=None earlier in create_provider). ProviderSettings is frozen with extra="forbid", so the attribute is always present on a real instance — the getattr default only ever fires for None.

Two problems. First, this is the only field read here without the provider-name guard every sibling branch uses (provider_settings is not None and provider_settings.name == "openai" at line 154, "claude" at 181, "hermes" at 213). Correctness rests entirely on registry.py:117 having nulled a mismatched object — invisible at this call site, and reachable precisely because the schema doesn't validate it (see the schema.py:2693 finding).

Second, getattr returns Any, which hides a genuine type error. Writing the guarded form directly fails ty today:

error[invalid-argument-type]: Expected `list[str] | None`,
  found `list[Literal["user", "project", "local"]] | None`

because list is invariant and the provider's parameter is widened to list[str]. This line is quietly suppressing the narrowing loss described in the type-widening finding below, and as a side effect a future rename of the schema field would silently disable the feature forever instead of raising.

Suggested change

	setting_sources=getattr(provider_settings, "setting_sources", None),
	setting_sources=(
	provider_settings.setting_sources
	if provider_settings is not None
	and provider_settings.name == "claude-agent-sdk"
	else None
	),

Fix the parameter type first (see the sibling finding on _resolve_skill_filter's narrowing), then use this guarded form.

[Jason Robert (jrob5756)]

RECOMMENDED

Two type issues the project's typechecker can't currently catch here.

1. -> list[str] | str. The installed SDK declares skills: list[str] | Literal["all"] | None. The str arm is pure width with no value behind it — the docstring says "the literal "all"," the tests assert == "all", and no other string is reachable. A future return "All" or return "project" would typecheck and then fail inside the CLI.

2. Narrowing lost at line 693. The schema's list[Literal["user", "project", "local"]] degrades to list[str] in the constructor and stays list[str] on self._setting_sources, with no boundary validation. The SDK forwards it unvalidated into argv (--setting-sources={','.join(...)}), so ClaudeAgentSdkProvider(setting_sources=["prject"]) puts a typo straight on the CLI command line. The PR's own tests construct the provider directly nine times, so this is reachable, not hypothetical.

One more thing worth flagging: make typecheck passing here says nothing, because line 49 binds ClaudeAgentOptions: Any = None in the ImportError fallback, which poisons the symbol and disables argument checking at the ClaudeAgentOptions(...) call site entirely. Against the real symbol, ty flags both skills= (line 1016) and setting_sources= (line 997) as invalid-argument-type.

Suggested change

	def _resolve_skill_filter(skill_names: list[str], setting_sources: list[str]) -> list[str] | str:
	def _resolve_skill_filter(
	skill_names: list[str], setting_sources: Sequence[str]
	) -> list[str] | Literal["all"]:

Carry the tier narrowing through the constructor too — declare a module-level SettingSource = Literal["user", "project", "local"] (or import the SDK's under TYPE_CHECKING) and type both the parameter at line 693 and self._setting_sources as list[SettingSource]. That also makes the getattr in factory.py unnecessary.

[Jason Robert (jrob5756)]

RECOMMENDED

Enabling project makes the CLI read <cwd>/.claude/settings.json, whose hooks entries run arbitrary shell commands on tool events, from a directory that's by design a target repo. Grepping every setting_sources reference in src/ turns up only docstrings, this assignment, and the option at line 997 — no logger call, no verbose_log, no event. Nothing in the run output distinguishes a run that loaded the target repo's hooks from one that didn't, and per the has_structured_config() finding, -v doesn't show it either.

This is out of step with how the repo handles comparable trust decisions elsewhere. config/validator.py's _report_dropped_components warns about plugin hooks/ precisely because a component that behaves differently inside a workflow than in the CLI is, in its own words, "exactly the silent divergence this feature exists to remove, so the difference is named before the run rather than discovered after it." That reasoning applies with more force here, since the plugin case drops the hooks and this one enables them. This file already has six logger.warning/logger.info call sites, including _warn_if_session_lookup_unavailable.

Suggested change

	self._setting_sources: list[str] = list(setting_sources or [])
	if self._setting_sources:
	logger.warning(
	"claude-agent-sdk: ambient settings tiers enabled (%s). The session will "
	"load settings, instructions and HOOKS from those tiers -- 'project' reads "
	"<working_dir>/.claude/settings.json, whose hooks run shell commands on "
	"tool events. Enable only for repositories trusted as much as the workflow.",
	", ".join(self._setting_sources),
	)

[Jason Robert (jrob5756)]

RECOMMENDED

self._setting_sources is set once per provider instance, and there's one instance per provider type. But the directory it resolves against is per agent — _resolve_session_cwd returns agent.working_dir or os.getcwd().

So setting_sources: [project], written for the one agent pointed at a target repo, also makes every other agent on that provider load .claude/settings.json — hooks included — from the directory conductor run was launched in. That's a wider blast radius than the field docstring (schema.py:2701-2706) describes, and it's the exact ambient-hook leakage AGENTS.md:434 cites as the reason the original invariant existed.

The field being workflow-global while working_dir is per-agent is the root cause, and it's the same mismatch behind the skills: [] finding above.

Suggested fix: either scope the setting per agent (an AgentDef-level field, so it can only apply where the workflow named a directory), or refuse the combination when an agent on this provider has no explicit working_dir, so a tier can never resolve against the launch directory by accident. At minimum, document the actual scope in the field docstring.

[Jason Robert (jrob5756)]

RECOMMENDED

class TestSettingSourcesWiring opens at line 2252, but TestMcpOptionsWiring was never closed first — everything from line 2406 to 2564 is now collected under the new class instead. Against origin/main, these six moved:

• test_config_file_removed_when_query_raises
• test_config_file_removed_on_interrupt_return
• test_no_config_file_leaks_when_options_construction_fails
• test_secrets_reach_the_file_but_not_the_options
• test_concurrent_executions_get_independent_config_files
• test_empty_tools_still_attaches_mcp_servers

TestMcpOptionsWiring went from 8 tests to 2. Nothing is orphaned or skipped — no class-scoped fixtures, all six still run, suite still reports 174 passing — which is exactly why CI won't catch this.

It still matters: test_secrets_reach_the_file_but_not_the_options and test_no_config_file_leaks_when_options_construction_fails are secret-leak and tempfile-leak regression guards. File them under a settings-sources heading and the next person doing MCP work greps TestMcpOptionsWiring, sees two tests, and assumes config-file cleanup is uncovered. The node IDs changed silently too, breaking any saved -k selection or test-ID reference.

Suggested fix: move the whole TestSettingSourcesWiring block (lines 2252-2404) down so it begins after line 2564, right before class TestMcpRequiredFields. Better still, place it after TestSkillsWiring — that's the class it logically extends, and it already provides _capture_options/_argv helpers these tests could reuse.