feat: user-defined LLM committees - #341
Open
duncanmcqueen wants to merge 2 commits into
Open
Conversation
Adds MIKE_MODEL_CONFIG_JSON so a deployment can declare a self-hosted or third-party OpenAI-compatible endpoint without a code change. Declared models resolve like any other id and are served by the existing AI SDK provider layer via @ai-sdk/openai-compatible, so they inherit its transport, retries and streaming rather than a second HTTP client. Endpoints that describe tool calls in prose instead of emitting them structured — routine in self-hosted Qwen, DeepSeek and GLM builds — are wrapped in a language-model middleware that routes <think> prose to the reasoning channel, suppresses tool markup in the visible text, and turns the described calls into real tool calls. A tolerant model answering a tool-declaring turn is served through the non-streaming endpoint, since these models interleave markup with prose in a way a partial stream cannot be reassembled from. Only OpenAI-compatible endpoints are declarable: the hosted providers are already covered by the static catalog and the router prefixes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A committee answers one prompt with several models and has a chair model synthesize their replies into the single answer the user sees. Committees are built under Settings > Models, stored on the profile, and selected from a Committee group in the model picker like any other model. Members run concurrently against the ordinary provider layer, so a committee can mix hosted and self-hosted models freely. Committee mode has no tool-calling loop: when the caller supplies tools the members are told so, rather than being left to claim document work they could not do. Availability is all-or-nothing. A committee is hidden from the picker and refused at request time unless every member and its chair is usable, and selecting a committee that has since been deleted is an explicit error — quietly answering with a different model misrepresents which model wrote the response. Cycles, self-chairing and nested committees are rejected on save and again at request time. Profile reads tolerate a database that has not applied the migration yet, reporting no committees rather than failing every chat and tabular request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
duncanmcqueen
force-pushed
the
feat/model-committees
branch
from
August 23, 2026 16:25
e0219db to
1789323
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds user-defined model committees: a committee answers one prompt with several
models at once and has a chair model synthesize their replies into the single
answer the user sees. Committees are built under Settings > Models, stored
on the profile (
user_profiles.model_committees), and selected from aCommittee group in the model picker like any other model.
Stacked on #339 — the first commit here is that PR's foundation. Retarget or
rebase once it merges.
What changed
Backend
lib/llm/committee.ts— members run concurrently against the ordinaryprovider layer, then the chair synthesizes. Guards against self-chairing,
self-membership and cycles.
lib/userCommittees.ts— normalization for reads (malformed rows aredropped, never thrown, so a bad row cannot lock a user out of Settings) and
strict validation for writes (max 8 committees, 2–8 members).
lib/llm/models.ts—missingCommitteeApiKeyModels/modelHasApiKey/resolveUsableModel: availability resolves recursively through acommittee's members and chair.
lib/routerModels.ts—resolveRequestedModelrecognises committee ids andgates them on availability, taking committees the caller already loaded so
no request reads the profile twice.
lib/userSettings.ts,lib/chat/streaming.ts,routes/chat.ts,routes/tabular.ts— committees are loaded once with the rest of theprofile and threaded to dispatch.
routes/user.ts—modelCommitteeson the profile (select / serialize /PATCH) plus
GET /user/models. Profile reads get their own retry tier, so adatabase without the column keeps every other live column.
migrations/20260823_02_user_model_committees.sql+schema.sql.Frontend
components/settings/CommitteeSettingsSection.tsx— create, edit and deletecommittees. Only models the user can actually run are offered as members.
ModelToggle— a Committee group, shown only when every member and thechair is usable.
mikeApi/UserProfileContext— committee types, profile state, save.Design notes
Availability is all-or-nothing. A committee is hidden from the picker and
refused at request time unless every member and its chair is usable. Half a
committee is not a usable answer.
A deleted committee is an explicit error, not a silent fallback to another
model — quietly answering with a different model misrepresents which model
wrote the response.
No tool-calling loop. When the caller supplies tools (the main chat path
always does) they are dropped and the members are told so, rather than being
left to claim document or case-law work they could not do.
Cost. Members are separate model calls, so a committee costs roughly the
sum of its members plus the chair. Running them concurrently keeps the latency
close to the slowest member instead of the sum.
Why
Different models disagree in useful ways on legal questions. A committee makes
that disagreement visible to a synthesizing model rather than making the user
run the same prompt several times by hand.
Testing
backend:tsc --noEmitclean;vitest run— 789 passed, 25 skipped(+25 over feat(llm): add model registry and OpenAI-compatible provider foundation #339's foundation, covering committee dispatch, concurrency,
cycle/self-reference guards, and committee validation).
frontend:tsc --noEmitclean;npm run lint0 errors;vitest run—564 passed, 14 failed. All 14 failures are pre-existing on
main(
mikeApi.test.tsBlob handling,useSelectedModel,ChatInput.modelSelection) and were confirmed against the baseline beforethese changes.
schema.sqlupdated to match.28 files changed in this commit (1681 insertions); 37 files and 3498 insertions
including #339's foundation.