[BETA] Desktop app MCP stdio layer#1649
Open
tanish2k09 wants to merge 12 commits into
Open
Conversation
Standalone crate at crates/cli/artcraft-mcp/ exposing a single MCP tool that submits an image generation via /v1/omni_gen/generate/image, polls /v1/jobs/session until the job completes, and returns the CDN URL. Reads Storyteller cookies from ~/Artcraft/credentials/ written by the desktop app — no separate sign-in needed. 90s poll timeout. Kept as its own [workspace] during the spike so the parent workspace Cargo.toml stays untouched and parallel work on main doesn't conflict. Workspace registration deferred to a final commit once the spike is validated end-to-end against Claude Desktop.
Production desktop v0.32 keeps Storyteller cookies only in the vendored tauri-plugin-http jar at <app_cache_dir>/.cookies; persist_all_to_disk() for the text-file path in the dev source is commented out, so shipped builds never populate those files. Add a JSON-parsing fallback that reads the .cookies jar, filters to api.storyteller.ai, and extracts the session + visitor cookies. The text-file path stays as a secondary fallback for forward-compat. macOS-only path resolution via directories::ProjectDirs for the "ai.artcraft.app" bundle id. Linux/Windows resolutions differ from Tauri's app_cache_dir() and are deferred.
The OmniGenImageCostAndGenerateRequest doc comment claims the server
auto-generates an idempotency_token when None is passed, but the
backend actually rejects requests without one ("idempotency token does
not match the expected format"). Match the desktop app's behavior at
handle_artcraft_via_omni_endpoint.rs:30 by sending a fresh hyphenated
v4 UUID per call.
Previously the tool returned only Content::text(cdn_url), which Claude Desktop renders as a "show image" link. Switch to fetching the generated image bytes from the CDN, base64-encoding them, and returning both Content::image and Content::text so the chat shows the image inline while keeping the URL clickable. On fetch failure (timeout, CDN error, etc.) we fall back to the URL-only response so the tool still succeeds.
Claude Desktop rejects tool results over 1 MB; base64 adds ~33% overhead so the raw payload must stay under ~750 KB. Full-res PNGs from Artcraft (often 1024+ px) routinely exceed that. Use media_links.maybe_thumbnail_template at width 768 for the inline preview when the API returns one; fall back to the full cdn_url otherwise. The full URL is still returned as text so the user can open the original at full resolution.
The tool now accepts an optional `model` arg (snake_case id matching CommonImageModel's serde rename, e.g. nano_banana_pro, flux_pro_1p1_ultra, gpt_image_2, midjourney_8, seedream_4, etc.). Defaults to nano_banana_pro when omitted. Unknown ids return a structured error so the LLM can correct itself. Valid model ids are listed in the JSON schema description so Claude can pick one before calling the tool.
Split main.rs (398 lines) into modules: main.rs, server.rs, creds.rs,
inline_image.rs, tools/{generate_image,list_image_models,
get_user_status,estimate_image_cost}.rs.
New tools:
- list_image_models — GET /v1/omni_gen/models/image; returns each
model's constraints (aspect_ratio_options, resolution_options,
quality_options, batch_size_max, image_refs_supported, image_refs_max,
text_prompt_max_length, etc.) so Claude can pick valid params before
calling generate_image. Optional provider filter.
- get_user_status — parallel GETs to /v1/credits/namespace/artcraft +
/v1/subscriptions/namespace/artcraft. Returns credits (free, monthly,
banked, total) and active subscription info.
- estimate_image_cost — POST /v1/generate/cost_estimate/image. Takes
model + num_reference_images + aspect_ratio + resolution + quality +
num_images. Returns cost_in_credits, cost_in_usd_cents, is_free,
is_unlimited, is_rate_limited, has_watermark — so Claude can confirm
with the user before spending credits.
Expanded generate_image:
- New optional params: aspect_ratio, resolution, quality, num_images,
reference_image_urls.
- reference_image_urls fetches each https URL, validates as image/*
(PNG/JPEG/GIF, max 20 MB), uploads via /v1/media_files/..., and
passes the resulting MediaFileToken to omni_gen. Marked as
intermediate system files so they don't appear in the user's gallery.
Constraint validation is left to backend's structured errors; Claude
is expected to consult list_image_models first.
Make failure modes legible to both the LLM and the user:
errors.rs:
- ErrorCode enum: artcraft_not_installed, not_logged_in,
session_expired, backend_unavailable, invalid_params,
generation_timeout, generation_failed, reference_image_failed,
internal.
- ToolError { error_code, message, remediation, details? } that
serializes as JSON in CallToolResult::error content blocks.
- not_logged_in() detects whether the app is installed
(/Applications/ArtCraft.app or the cache dir) and returns the
appropriate code + remediation text.
creds.rs:
- load_session() now returns Result<_, ToolError>. The user sees a
proper sign-in remediation instead of a generic anyhow message.
tools/check_artcraft_connection.rs (new):
- Always succeeds, never errors. Reports desktop_app_installed,
signed_in, session_valid, error_code, remediation. Detects expired
sessions by making a cheap authenticated get_session_credits call.
tools/list_image_models.rs:
- 401/403 from the API now maps to session_expired.
All existing tools converted from anyhow::Result to Result<_, ToolError>
with appropriate error codes per failure path.
server.rs:
- Updated instructions teach Claude how to react to each error code
(open desktop app and sign in, download from getartcraft.com, etc.)
and to call check_artcraft_connection when troubleshooting.
- error_result() now serializes the full ToolError struct as JSON
inside the error content block.
main.rs:
- New '--check' / '-c' CLI flag prints a human-readable diagnostic to
stdout and exits, useful when the user can't see the Artcraft tools
in Claude and wants to debug from a terminal without round-tripping
through MCP.
When num_images>1, the omni_gen response is still a single job with a
single inline result; the rest of the batch is reachable through the
result's maybe_batch_token via GET /v1/media_files/batch_gen_redux/{token}.
Previously we only surfaced the single inline result, so 'num_images: 4'
returned just 1 image. Now after poll detects CompleteSuccess we follow
the batch token to fetch every member and return them all.
Inline rendering:
- Single result: 768 px thumbnail (unchanged).
- Batch: 512 px per image — 4 thumbnails at 512 px JPEG stay well under
Claude Desktop's 1 MB tool-result cap.
Per-image inline fetch failures are logged and skipped (still return
the URL as text), so a single bad thumbnail doesn't poison the whole
response.
Multi-image text output is one URL per line, prefixed 'Image N:'.
Drop the standalone [workspace] block from the crate's Cargo.toml and add 'crates/cli/artcraft-mcp' to the root workspace members list. Kept this as the final commit on the branch — until this lands the crate was a separate workspace, which kept the root Cargo.toml untouched during the spike (avoiding conflicts with parallel work on main) but meant 'cargo build' at the repo root didn't include it. After this commit, 'cargo check -p artcraft-mcp' works from the repo root and the crate is a first-class workspace member.
Records rmcp + transitive deps added when the crate joined the parent workspace.
Lets the LLM truthfully answer 'what did that just cost?' without
remembering the pre-flight estimate. The backend has no per-job cost
endpoint or credit-transactions API; the only way to know the realized
cost is to diff the credit balance before and after.
generate_image::run now:
- Captures get_session_credits before submit.
- Runs generation + poll as before.
- Captures get_session_credits after success.
- Computes credits_charged = balance_before - balance_after.
- Attaches { credits_charged, credits_balance_after } to the response.
Best-effort: if either credit lookup fails the generation still
succeeds and the cost just isn't reported. Caveat noted in comments:
concurrent generations settling inside this window can over-count the
diff.
The tool's text content gains a 'Cost: N credits charged. Balance
after: M credits.' line when cost info is available — picked up by
both the user reading the chat and the LLM answering follow-up
questions about cost.
✅ Deploy Preview for artcraft-website ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for genhub-website failed. Why did it fail? →
|
✅ Deploy Preview for artcraft-webapp ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for storyteller-docs failed. Why did it fail? →
|
Collaborator
Author
|
@echelon code touches rust, requesting review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incomplete deep support for MCP features for external LLM providers like Claude based on local desktop data: