Add Langfuse observability integration

[shatfield4]

Pull Request Type

• ✨ feat (New feature)
• 🐛 fix (Bug fix)
• ♻️ refactor (Code refactoring without changing behavior)
• 💄 style (UI style changes)
• 🔨 chore (Build, CI, maintenance)
• 📝 docs (Documentation updates)

Relevant Issues

resolves #

Description

• add admin Observability settings page for connecting an external tracing platform (Langfuse is the first provider we support with infra for adding other providers in the future)
• trace every inference path: workspace/thread chats, developer API, embed widget, OpenAI-compatible endpoint, telegram, agents (with tool call spans), document + query embeddings, image generation
• trace internal LLM calls with usage + cost: content summarization (web-scraping/summarize skills), agent flow LLM blocks, memory extraction, model router classification, pptx builder sub-agents; TTS/STT calls traced with timing + payload sizes
• fix pptx section sub-agents silently failing to build slides (stale handleAsyncExecution call signature always threw + fell back to stub slides)
• report token usage and cost on every trace using the app's own model pricing (chats, agents, paid embedders)
• attribute every trace to the acting user and conversation session in both single and multi user mode
• ship traces over Langfuse's OpenTelemetry endpoint with zero new dependencies, fully async - an offline observability endpoint never blocks or fails a chat
• flush pending traces before background job processes exit so worker-run embeds are never lost

Visuals (if applicable)

Additional Information

• Add docs for observability feature

Known cost/token tracking gaps

• Tool call child spans on agent traces carry no usage/cost - spend is aggregated on the parent generation
• Providers with no pricing source (generic-openai, litellm, and other aggregators/proxies) never get a cost
• TTS/STT and image generation traces have timing/size metadata but no cost (no per-character/minute/image pricing source)
• Token counts are tiktoken estimates when the provider does not report usage (always the case for embeddings)
• Agent traces show 0 usage for providers that do not report usage chunks

Testing

Single user mode

• Workspace chat (UI) - with vector-search span + citation sources
• Workspace chat (dev API, sync + stream) - custom sessionId honored
• Thread chat (dev API) - session falls back to workspace:thread
• OpenAI-compatible endpoint (sync + stream)
• Embed widget
• Agent chat (UI websocket) - tool call child spans, cumulative usage, cost
• Agent chat (dev API ephemeral)
• Document embedding - API in-process path (embedding-ingest, chunk count + workspace)
• Document embedding - UI background-worker path (flush-before-exit verified)
• RAG query embedding on every chat (embedding-query)
• Paid embedder cost - OpenAI text-embedding-3-small: ingest + query traces carry real $ cost
• Local embedder
• Image generation (/img) - provider, model, duration, file size (no cost tracking yet for img providers)
• Langfuse UI renders traces, costs, sessions, spans correctly
• Telegram - regular + agent chats, attributed to telegram:<username>, agent traces tagged with telegram origin
• Thread sessions - agent chats share the same workspace:threadSlug session as regular chats in that thread
• Content summarization (summarize) - chunked token usage + real $ cost (forced via small-context model + large page scrape)
• Memory extraction job - observer + reflector generations with usage/cost, no duplicate anonymous agent-chat traces
• Agent flow LLM instruction block - usage + cost
• Model router LLM classification - usage + cost, no duplicate agent-chat trace
• pptx section sub-agent - success and error paths both trace with cumulative usage
• TTS + STT - timing + payload size metadata
• No broken functionality if Langfuse endpoint is down

Multi user mode

• All of the above dev API + embed + OpenAI-compatible paths still pass
• Workspace & agent chats attributed by user
• Document embedding (UI worker upload) attributed to acting user
• RAG query embedding attributed to acting user (admin, manager, default)
• Image generation attributed by user
• Observability settings admin only page

Developer Validations

• I ran yarn lint from the root of the repo & committed changes
• Relevant documentation has been updated (if applicable)
• I have tested my code functionality
• Docker build succeeds locally