You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(readme): describe the landed AI stack, not the stub it replaced
Chat streaming, typed extraction, the shared InjectableNativeLlm seam,
BYOK settings/desktop and the toolkit 2.0.0 pin had all merged but the
README still described the old separate seams, an untyped OCR scrape,
and stale toolkit/test/screenshot counts.
- 🤖 **On-device LLM assistant, one shared seam, streaming.** The expense chat streams real
128
+
per-token replies behind `LlmGateway`, ML Kit GenAI (Gemini Nano) on Android, Apple Foundation
129
+
Models on iOS through one shared Swift bridge, kmp-toolkit's `InjectableNativeLlm`/`OnDeviceLlm`
130
+
seam that document extraction above now runs on too, degrading to the offline retrieval engine
131
+
wherever no model is available. AI is visible in Settings and on desktop: consent, on-device model
132
+
download/pause/delete, and a BYOK cloud key (Anthropic/OpenAI/Gemini, tested and stored via
133
+
`SecureKeyStore`) as the fallback tier everywhere, the *only* answer path on desktop, which has no
134
+
on-device model at all. Neither on-device tier is device-verified here: no Gemini-Nano-class
135
+
hardware (Pixel 8+/AICore-eligible) and no Apple Foundation Models device in this environment, so
136
+
both are compile-verified only.
126
137
127
138
## Screenshots
128
139
@@ -421,7 +432,7 @@ Doori/
421
432
| Charts | Canvas-only (no MPAndroidChart / Vico) |
422
433
| Theming | MaterialKolor **5.0.0**|
423
434
| Capture | Peekaboo (KMP camera/gallery) |
424
-
| On-device AI | ML Kit GenAI (Android) / Apple Foundation Models (iOS, Swift-bridge) behind a shared `LlmGateway`, text recognition + barcode scanning, degrading to an offline heuristic engine where a model isn't available |
435
+
| On-device AI |kmp-toolkit's `InjectableNativeLlm`/`OnDeviceLlm` seam, ML Kit GenAI (Gemini Nano, Android) / Apple Foundation Models (iOS, one shared Swiftbridge) behind `LlmGateway` (streaming chat) and `DocumentAiAnalyzer` (typed extraction); a BYOK cloud fallback (Anthropic/OpenAI/Gemini via `SecureKeyStore`) on every platform including desktop, which has no on-device tier at all; text recognition + barcode scanning; degrading to an offline heuristic engine where no model is available. Neither on-device tier is device-verified here (no Gemini Nano / Foundation Models hardware)|
@@ -609,8 +620,10 @@ roadmap reflects direction rather than commitments.
609
620
-[x]**AI assistant / "agent" feature (V20).** Offline, retrieval-grounded chat over real local
610
621
trip/expense/card data; Room-backed persistent history + 5-minute session resume; on-device
611
622
voice I/O (STT/TTS); feedback, export and real-usage popular-question ranking; full
612
-
`commonMain` + iOS parity. (A dedicated Popular/Unanswered analytics screen and persisted
613
-
unanswered-question submission are still open, tracked as backlog.)
623
+
`commonMain` + iOS parity. The dedicated Popular/Unanswered analytics screen (`PopularTab`,
624
+
`UnansweredTab` in `AgentChatScreen`) has since shipped; persisted unanswered-question
625
+
submission has not, `AgentRepository.unansweredQuestions` still reads from
626
+
`AgentMockData`, not Room, tracked as backlog.
614
627
-[x]**On-device LLM backing (post-V25).**`LlmGateway` swaps the assistant onto a real on-device
615
628
model, ML Kit GenAI on Android, Apple Foundation Models on iOS (Swift bridge,
616
629
`xcodebuild`-gated, not yet device-verified), degrading to the offline retrieval engine
@@ -686,6 +699,20 @@ roadmap reflects direction rather than commitments.
686
699
-[x]**iOS launch-crash fix (V33).**`CADisableMinimumFrameDurationOnPhone` added to `Info.plist`
687
700
(a Compose Multiplatform `PlistSanityCheck` requirement), the iOS app builds
688
701
(`xcodebuild`-green) and launches correctly on device.
702
+
-[x]**AI stack unification (V35).** Chat and document extraction share ONE on-device seam now:
703
+
kmp-toolkit's `InjectableNativeLlm`/`OnDeviceLlm` (`MlKitGenAiOnDeviceLlm` on Android,
704
+
`FoundationModelsOnDeviceLlm` over a single Swift `FoundationModelsBridge` registered once in
705
+
`AppDelegate.swift` on iOS), replacing this app's own separate seams. `feature:agent`'s
706
+
`LlmAssistantEngine` streams real per-token output through it; `core:ai`'s
707
+
`DocumentAiAnalyzer` returns a typed `AiResult<AiExtraction>` via `structuredOutput<...>()`
708
+
instead of a hand-rolled regex scrape, and OCR-to-form prefill is unified on one
709
+
`FormFieldWithSuggestions` component. AI is now visible in Settings (consent, on-device model
710
+
download/pause/delete, BYOK cloud-key entry and test) on Android and on the Compose Desktop
711
+
target, which has no on-device model at all and always answers through the BYOK cloud
712
+
fallback (Anthropic/OpenAI/Gemini keys via `SecureKeyStore`). **Not device-verified:**
713
+
Gemini-Nano-class hardware (Pixel 8+/AICore-eligible) and an Apple Foundation Models device
714
+
are both unavailable in this environment, so both on-device tiers remain compile-verified
715
+
only, same caveat the V25 bullet above already carried.
689
716
690
717
**Exploring**
691
718
@@ -703,6 +730,9 @@ roadmap reflects direction rather than commitments.
703
730
`NetworkBackendFlags.useRealBackend` is a compile-time `false` with no debug toggle wiring it
704
731
on, so the authenticated path has never run in an actual app process, that wiring, plus the
705
732
remaining PLAN_V33.1 routes beyond miles/location/events, is the next step
733
+
-[ ] On-device LLM hardware verification: Gemini Nano (Pixel 8+/AICore-eligible) and Apple
734
+
Foundation Models are both compile/build-verified only here, no eligible device in this
735
+
environment
706
736
707
737
## iOS, Wear OS and watchOS
708
738
@@ -747,6 +777,7 @@ roadmap reflects direction rather than commitments.
747
777
| iOS WidgetKit + Live Activity/Dynamic Island | ✅ `xcodebuild -scheme MilewayWidgets build`| ✅ host-rendered screenshots (WidgetScreenshotTests) | ✅ widgets + Live Activity captured |
748
778
| iOS App Intents / Siri Shortcuts | ✅ compiles, `AppShortcutsProvider` registered |, | ⏸ Siri phrase invocation needs a device/simulator with Siri running |
| On-device LLM (chat streaming + document extraction) | ✅ compiles both platforms | ✅ unit-tested (`LlmAssistantEngineTest`, `DocumentIntelligenceTest`) | ⏸ Gemini Nano and Apple Foundation Models both need eligible hardware, none available here; the BYOK cloud fallback is the only tier ever exercised live |
0 commit comments