feat: pluggable storage adapter — R2, GCS, S3-region config - #47
Open
amal66 wants to merge 1 commit into
Open
Conversation
Route all storage operations through a StorageAdapter interface. The default R2StorageAdapter preserves existing R2 behavior exactly (same env vars, same bucket default, same error handling); GCSStorageAdapter adds Google Cloud Storage via Application Default Credentials. setStorageAdapter() swaps backends without touching call sites. The S3 region is now configurable via R2_REGION (default "auto", unchanged). Path/filename helpers move to core/storagePaths.ts and are re-exported from lib/storage, so every existing import keeps working. Mechanical port of the storage adapter work from amal66/mike@main (commit b3166dd), translated to the upstream backend/ layout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
amal66
added a commit
that referenced
this pull request
Aug 21, 2026
…in the app) WHY THIS MATTERS The hosted default makes a downloaded Mike.app usable immediately, but it makes the app a client of the cloud. Some users (law firms with data- residency rules, air-gapped environments, privacy-first individuals) want the opposite: everything on their own machine, with the same one-click download. Today that means running a docker-compose stack — a non-starter for non-developers. This document is the researched plan for folding that whole stack INSIDE the app. WHAT IS A "SELF-CONTAINED" DESKTOP APP Instead of the Electron shell loading a remote URL, the app bundles and supervises its own local services — database, auth, API, storage — all on loopback, and points the window at http://localhost. The user never knows a server exists. The key design constraint carried over from the shell: the web app and backend run byte-identical to the compose stack (shell- over-rewrite); only the process manager changes from docker-compose to a supervisor inside the app. WHAT THE RESEARCH FOUND (the plan is cheaper than it looks) - Postgres needs only pgcrypto + pg_trgm — no pgvector on main. - The frontend touches Supabase ONLY for auth; all data flows through the backend, whose sole DB path is PostgREST (436 call sites). So bundling the real GoTrue + PostgREST + Postgres binaries (all permissively licensed, all arm64-mac-buildable) preserves every API contract with zero rewrites. - No Redis/queues on main; document conversion degrades gracefully without LibreOffice; storage already has an adapter seam in flight (upstream PR #47 lineage) that a local-filesystem driver can plug into. - LLM access already supports per-user keys and local Ollama; the keyless demo mode (PR open-legal-products#260) covers the first-run answer. HOW THE PLAN IS STRUCTURED Five phases, each independently shippable: (0) upstream seams as normal web-app PRs — storage adapter + fs driver, migration runner, Next standalone flag; (1) a local-stack supervisor in the shell with a "Run locally on this Mac" mode on the connect screen; (2) packaging + signing of the bundled binaries; (3) upgrade/backup lifecycle; (4) first-run polish. Roughly 6-9 engineering weeks. Open product decisions (auth UX, password recovery without SMTP, LibreOffice, Intel support) are listed explicitly rather than silently decided. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
amal66
added a commit
that referenced
this pull request
Aug 25, 2026
…in the app) WHY THIS MATTERS The hosted default makes a downloaded Mike.app usable immediately, but it makes the app a client of the cloud. Some users (law firms with data- residency rules, air-gapped environments, privacy-first individuals) want the opposite: everything on their own machine, with the same one-click download. Today that means running a docker-compose stack — a non-starter for non-developers. This document is the researched plan for folding that whole stack INSIDE the app. WHAT IS A "SELF-CONTAINED" DESKTOP APP Instead of the Electron shell loading a remote URL, the app bundles and supervises its own local services — database, auth, API, storage — all on loopback, and points the window at http://localhost. The user never knows a server exists. The key design constraint carried over from the shell: the web app and backend run byte-identical to the compose stack (shell- over-rewrite); only the process manager changes from docker-compose to a supervisor inside the app. WHAT THE RESEARCH FOUND (the plan is cheaper than it looks) - Postgres needs only pgcrypto + pg_trgm — no pgvector on main. - The frontend touches Supabase ONLY for auth; all data flows through the backend, whose sole DB path is PostgREST (436 call sites). So bundling the real GoTrue + PostgREST + Postgres binaries (all permissively licensed, all arm64-mac-buildable) preserves every API contract with zero rewrites. - No Redis/queues on main; document conversion degrades gracefully without LibreOffice; storage already has an adapter seam in flight (upstream PR #47 lineage) that a local-filesystem driver can plug into. - LLM access already supports per-user keys and local Ollama; the keyless demo mode (PR open-legal-products#260) covers the first-run answer. HOW THE PLAN IS STRUCTURED Five phases, each independently shippable: (0) upstream seams as normal web-app PRs — storage adapter + fs driver, migration runner, Next standalone flag; (1) a local-stack supervisor in the shell with a "Run locally on this Mac" mode on the connect screen; (2) packaging + signing of the bundled binaries; (3) upgrade/backup lifecycle; (4) first-run polish. Roughly 6-9 engineering weeks. Open product decisions (auth UX, password recovery without SMTP, LibreOffice, Intel support) are listed explicitly rather than silently decided. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Route every storage operation through a small
StorageAdapterinterface so the object-storage backend can be swapped without touching call sites. Cloudflare R2 stays the default and behaves exactly as before; a new Google Cloud Storage adapter (ADC auth) can be installed with onesetStorageAdapter()call at startup. The S3 client region becomes configurable viaR2_REGION(default"auto", unchanged).Changes
backend/src/lib/storage/adapter.ts— theStorageAdaptercontract (upload / download / delete / list / getSignedUrl / checkReady).backend/src/lib/storage/r2.ts—R2StorageAdapter: the existing R2 code moved behind the interface. Same env vars, same"mike"bucket default, same error messages and null/empty fallbacks; region now readsR2_REGION(default"auto", the previous hardcoded value).backend/src/lib/storage/gcs.ts—GCSStorageAdapter: Google Cloud Storage via Application Default Credentials (GCS_BUCKET_NAME,GCS_PROJECT_ID,GCS_SIGNED_URL_TTL).backend/src/lib/storage.ts— now a thin public API: delegatesuploadFile/downloadFile/deleteFile/listFiles/getSignedUrlto the active adapter, addssetStorageAdapter()+checkStorageReady(), and re-exports everything callers already import.backend/src/core/storagePaths.ts— the pure path/filename helpers (storageKey,buildContentDisposition, …) moved out ofstorage.ts; re-exported fromlib/storageso no import changes anywhere.backend/tsconfig.json— excludesrc/**/__tests__/**and*.test.tsfrom the build (tests run under vitest, not tsc).backend/src/lib/__tests__/storage.test.ts(25 tests, path/filename helpers),backend/src/lib/storage/__tests__/gcs.test.ts(16 tests, GCS adapter with a mocked SDK).backend/package.json/package-lock.json— add@google-cloud/storage@^7.19.0.Why
Deploying anywhere other than Cloudflare requires either S3-compatible credentials or a fork of
storage.ts. The adapter makes the backend portable: GCS out of the box, and any other backend (Supabase, local FS, S3) by implementing one interface. New runtime dependency:@google-cloud/storage@^7.19.0(already shipped by the fork). Zero behavior change unless configured: the default adapter is R2 with identical env vars, defaults, and fallbacks — verified by reading each method against the previousstorage.ts;R2_REGIONand the GCS vars only take effect when explicitly set.Testing
npm install && npm run build(tsc) green on the branch as committed, from a cleannode_modules.upstream-pr/test-harness) merged locally:npm test→ 3 files, 53 tests passed (25 storage + 16 gcs from this branch, 12 pre-existing downloadTokens).Provenance
All changes are mechanical ports of amal66/mike@origin/main (commit b3166dd):
apps/api/src/lib/storage/{adapter,gcs,r2}.ts→backend/src/lib/storage/,apps/api/src/lib/storage.ts→backend/src/lib/storage.ts,apps/api/src/core/storagePaths.ts→backend/src/core/storagePaths.ts, plus their tests. Exceptions: (1) the fork reads config through its zodlib/envmodule, which this base lacks — env reads were translated to directprocess.envaccess with the fork's defaults (R2_BUCKET_NAME/GCS_BUCKET_NAME→"mike",R2_REGION→"auto",GCS_SIGNED_URL_TTL→3600), and the two tests'vi.mock("../env")blocks became plainprocess.envset/delete; (2) the tsconfig test-exclusion line, matching the other upstream-pr branches.Credits & prior art
StorageAdapterinterface with R2 behavior preserved verbatim), same portability need.🤖 Generated with Claude Code
https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC