Ecash migration module #7976

elsirion · 2025-11-21T11:17:26Z

WARNING: heavy changes still ahead, especially commit history is a bit messy. General consensus architecture should be solidifying now though. A rough outline below:

elsirion · 2025-12-15T17:26:23Z

Still need to do a final review myself, approach was:

Claude: First version
Manual: fixups and changes we discussed on the review call (merkle tree impl)
Claude: add documentation
Claude: split into reasonable commits, making sure that the change set stays the same
Claude: Rebase on master

I think it's ready for review, getting AI review in parallel now.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

…rastructure Implement core server-side functionality for ecash migration: - Byzantine fault-tolerant consensus for migration transfers - Spend book upload mechanism with authentication - Database schema for transfers and spend book entries - API endpoints for transfer management and upload progress tracking - Code organization improvements (api module, db utilities) This provides the foundation for secure, federated ecash migration.

Implement comprehensive client-side functionality for ecash migration: - Register and manage transfers - Upload spend book data using merkle proofs - Fund transfers with ecash notes - Request activation of migration transfers - Redeem migrated funds This implementation uses merkle tree proofs for efficient and verifiable spend book uploads, replacing the initial naive threshold approach.

Replace unwrap() calls with expect() with clear messages explaining why the operations cannot fail, following project code quality standards.

Convert logging statements to structured format (field = value) as per project standards in the upload_spend_book function.

Add runtime validation to ensure spend book nonces are in strictly ascending sorted order. This prevents silent failures where Merkle tree construction depends on consistent ordering. Panics with a clear error message if sortedness is violated.

Replace inconsistent use of 'liability transfer' with 'transfer' throughout documentation and comments for simplicity and consistency. The module name 'ecash-migration' provides sufficient context.

elsirion · 2025-12-22T17:21:11Z

@elsirion update diagram

dpc · 2025-12-23T21:23:45Z

modules/fedimint-ecash-migration-common/src/naive_threshold.rs

+pub struct NaiveThresholdSignature {
+    /// A concatenated list of signatures. Has to contain at least the threshold
+    /// number of signatures. The remaining signatures can be `None`.
+    signatures: Vec<Option<Signature>>,


Would it make sense to enforce this being ordered, so it is deterministic?

I don't get why Option is needed. Can't we just assume ones that are not in a vec, are None?

Oh, I see, this have to be the same len() as pubkeys and correspond one to one. This is not immediately clear. I would go for HashSet<usize, Signature> which is slightly more compact, but it just a matter of preference, I guess.

dpc · 2025-12-23T21:26:40Z

modules/fedimint-ecash-migration-common/src/naive_threshold.rs

+/// A naive threshold signature scheme public key containing all public and the
+/// threshold number of signatures required to verify a signature.
+#[derive(Debug, Clone, Eq, PartialEq, Hash, Deserialize, Serialize, Encodable, Decodable)]
+pub struct NaiveThresholdKey {


Add Public?

dpc · 2025-12-23T21:54:03Z

modules/fedimint-ecash-migration-common/src/naive_threshold.rs

@@ -0,0 +1,148 @@
+//! Naive threshold signature scheme implementation using concatenated Schnorr


Probably could be a separate crate under ./crypto/. More crates = better.

dpc · 2025-12-23T21:56:32Z

modules/fedimint-ecash-migration-server/src/db.rs

+#[derive(Debug, Clone, Encodable, Decodable, Serialize, Deserialize)]
+pub struct TransferMetadata {
+    /// Pre-committed spend book hash (set at creation time)
+    pub origin_spend_book_hash: SpendBookHash,


I don't know if I'm just slow and it was obvious for everyone else, but I just realized that since spend book contains only used nonces, the guardians assuming the liabilities need to explicitly track how many nonces were redeemed per spend book, because the origin guardians could have dumped on them a spent book for which they inflated or they are even still inflating after it was transferred.

dpc · 2025-12-23T22:16:27Z

modules/fedimint-ecash-migration-common/src/lib.rs

 /// Modules are non-compatible with older versions
 pub const MODULE_CONSENSUS_VERSION: ModuleConsensusVersion = ModuleConsensusVersion::new(0, 0);

+/// Unique identifier for an ecash migration transfer


Unique how exactly? Sequential?

dpc · 2025-12-23T22:20:34Z

modules/fedimint-ecash-migration-server/src/lib.rs

+        Ok(())
+    }
+
+    async fn upload_spend_book_batch(


So in no way is this authenticated and until we get all entries we can't tell if there was no "corruption"? :/

dpc · 2025-12-23T22:38:59Z

modules/fedimint-ecash-migration-server/src/lib.rs

    ) -> Result<InputMeta, EcashMigrationInputError> {
-        Err(EcashMigrationInputError::NotSupported)
+        let (transfer_id, note, amount) = match input {
+            EcashMigrationInput::RedeemOriginEcash {


I see, so there are special inputs that are accounted and checked against a balance of each transfer. Nice.

dpc · 2025-12-23T22:41:06Z

modules/fedimint-ecash-migration-server/src/lib.rs

+            .await
+            .is_some();
+        if is_spent_on_origin || is_spent_on_local {
+            return Err(EcashMigrationInputError::AlreadyRedeemed(note.nonce));


Would split this error, because there's an important difference and caller might want to know which one was it.

dpc · 2025-12-23T22:42:18Z

modules/fedimint-ecash-migration-server/src/lib.rs

+            &WithdrawnAmountKey(*transfer_id),
+            &withdrawn_amount
+                .checked_add(*amount)
+                .ok_or(EcashMigrationInputError::Overflow)?,


expect? I mean ... this can't really happen if not a fatal bug.

dpc · 2025-12-23T22:52:27Z

modules/fedimint-ecash-migration-server/src/db.rs

+
+/// Prefix for querying all deposited amounts
+#[derive(Debug, Clone, Encodable, Decodable, Serialize)]
+pub struct DepositedAmountPrefix;


Any reason not to consolidate deposit and withdrawal into one record? Generally it seems like one transfer_id -> everything_singular_about_that_transfer_id might be just right? Seems like the only concern would be maybe write-write conflicts with any API calls.

dpc · 2025-12-23T22:57:05Z

modules/fedimint-ecash-migration-server/src/lib.rs

+            api_endpoint! {
+                GET_TRANSFER_ID_ENDPOINT,
+                ApiVersion::new(0, 0),
+                async |module: &EcashMigration, context, request: OutPoint| -> TransferId {


I was just thinking that for UX purposes would be good to have a way to look up transfer_id(s) via (self-reported during transfer creation, optional, advisory) FederationId of whos liabilities were transfered. This way clients could potentially easily detect that one federation has funds of some other federation, even without external coordination.

dpc · 2025-12-23T23:31:24Z

modules/fedimint-ecash-migration-server/src/lib.rs

+    async fn get_transfer_id(
+        &self,
+        dbtx: &mut DatabaseTransaction<'_>,
+        out_point: OutPoint,


I still don't know what is this OutPoint and I don't like it anyway. Couldn't we key requests like this on spend_book_hash or something? That should be unique already.

dpc · 2025-12-23T23:34:36Z

modules/fedimint-ecash-migration-server/src/lib.rs

+    TransferId(max_id + 1)
+}
+
+async fn check_auth(


The name could be longer and more specific.

Oh, I see, so this is to auth the requests. This should return some RequestAuthToken that handler functions would take, to make forgetting about auth structurally harder via type system.

dpc · 2025-12-23T23:36:59Z

modules/fedimint-ecash-migration-server/src/lib.rs

+        .ok_or(ApiError::not_found("Transfer not found".to_owned()))?;
+    let creator_keys = transfer_metadata.creator_keys;
+
+    let auth = context.request_auth().ok_or(ApiError::unauthorized())?;


Oh... so we ... use the request_auth ... Yyyy... Feels like mixing different concerns etc. and asking for trouble down the line. Would rather take the signature as a separate argument. Could be struct SignedRequest<RequestT> { inner: RequestT, signature: NaiveThresholdSignature } for convenience.

dpc · 2025-12-23T23:38:46Z

fedimint-dbtool/src/lib.rs

        .into_database()
 }
+
+async fn open_db_ro(options: &Options) -> fedimint_core::db::Database {


Weird one-of. Shouldn't we either already have this or use it in more places right away?

dpc · 2025-12-23T23:47:30Z

modules/fedimint-ecash-migration-client/src/lib.rs

+///
+/// # Errors
+/// - If the origin config file cannot be read or parsed.
+pub async fn read_origin_keyset(


Shouldn't this be server side command? Because later down the line we might do migrations and change stuff and then matching client version to server version will be a pain, while it's easy to tell guardian to use the same binary they were using for running the federation in the first place, no? IIRC joshisan wanted to eventually move configs into db, or the other way around - point being that potentially these are moving targets.

We could have some fedimintd export-post-mortem --data-dir ... that would create some versioned (so we can change the format) "export file", and possibly an guardian-authed api endpoint that can do the same thing, or something, and in the client we would be reading the standardized format only, not snoop in the server internals.

dpc · 2025-12-24T00:06:48Z

modules/fedimint-ecash-migration-client/src/lib.rs

+    }))
+}
+
+/// Build Merkle root from a spend book file using streaming with O(log n)


Reviewing Quest (note for myself): what do we need merkle tree for... ?

dpc · 2025-12-24T00:11:23Z

modules/fedimint-ecash-migration-client/src/lib.rs

+}
+
+/// Upload a single chunk to all peers with retries.
+async fn upload_chunk_to_peers(


_to_guardians ?

dpc · 2025-12-24T00:16:39Z

modules/fedimint-ecash-migration-client/src/lib.rs

+    KeySetHash(keyset.consensus_hash_sha256())
+}
+
+/// Open a spend book file and return a stream of nonces.


Seems like a very raw ah-hoc format, without versioning or ability to add helpful metadata etc.

Though if we formalize it into something, we probably want to retain ability to stream the output, which is not always easily doable with standard encoding libraries.

We might do some midle-ground, and e.g. make it text file, that looks a bit like http request

version: 0 key1: value1 key2: value --- <lines with nonces>

or something. Not a big complication really, but would make it somewhat more flexible. The header part could be even a single line of json.

That's like biggest life lesson I have to always add version to every protocol and data format one creates. :D

dpc · 2025-12-24T00:18:53Z

modules/fedimint-ecash-migration-client/src/lib.rs

+    ///
+    /// # Errors
+    /// - If the operation does not exist.
+    pub async fn subscribe_register_transfer(


Any eventlog events we should emit?

dpc · 2025-12-24T00:25:58Z

modules/fedimint-ecash-migration-client/src/lib.rs

+            }))
+    }
+
+    /// Get the controller keypair derived from the module secret.


The "controller" here isn't immediately self-explanatory and seems kind of generic. Maybe transfer_coordinator or something?

I wonder if something in here could be useful:

BTW. We probably should have some formal way to shut down federation for good. Could be a vote on consensus itself with signatures, plus ability to present a detached closure document. Just so the clients can be detect and inform users that the Federation is now official closed, possibly with some pointers on what to do about the assets. Different PR etc. but just thinking aloud.

dpc · 2025-12-24T00:36:36Z

modules/fedimint-ecash-migration-client/src/lib.rs

+
+    /// Upload the spend book entries in chunks to the destination federation.
+    ///
+    /// Each chunk is accompanied by a Merkle proof that allows the server to


Do we really need a merkle tree? Our case seems much simpler. Controllers could just prepare a list of threshold sigs for each batch of nonces, and maybe even we could just hardcode a batch size (to aim at like 1MB chunks), so nonces are uplaoded in signed batches. It's not really server problems to verify a signed batch belongs to the root. If it doesn't, it is sender's problem that they wasted the fee for uploading corrupted data.

dpc · 2025-12-24T00:38:44Z

modules/fedimint-ecash-migration-common/src/merkle.rs

@@ -0,0 +1,553 @@
+//! Merkle tree implementation for streaming data.


Could be a separate crate if really needed. I'm skipping reviewing because I'm not sure if we really need it, and this PR is already massive. :D

dpc · 2025-12-24T00:39:36Z

modules/fedimint-ecash-migration-common/src/naive_threshold.rs

@@ -1,148 +0,0 @@
-//! Naive threshold signature scheme implementation using concatenated Schnorr


Ohhhh... That's why merkle tree is used. Because we got rid of threshold sigs ... . So this is now our auth...

dpc · 2025-12-24T00:57:13Z

Need to review the merkle tree thing, and continue reviewing from "test: add ecash liability migration test"

elsirion force-pushed the 2025-11-ecash-migration branch 2 times, most recently from 8a80eef to 2aac10b Compare December 15, 2025 17:18

elsirion marked this pull request as ready for review December 15, 2025 17:18

elsirion requested review from a team as code owners December 15, 2025 17:18

elsirion changed the title ~~WIP: ecash migration module~~ Ecash migration module Dec 15, 2025

elsirion requested a review from Copilot December 15, 2025 17:19

elsirion force-pushed the 2025-11-ecash-migration branch from 2aac10b to 7661025 Compare December 15, 2025 17:34

Copilot AI reviewed Dec 15, 2025

View reviewed changes

elsirion added 4 commits December 19, 2025 15:42

chore: make copy of empty module

2d477a6

feat(dbtool): dump spend book from server db

c3e2da8

elsirion force-pushed the 2025-11-ecash-migration branch from 1631822 to d04707c Compare December 19, 2025 14:50

elsirion added 6 commits December 20, 2025 17:53

test: add ecash liability migration test

289d78b

refactor(ecash-migration): replace unwrap() with expect() in merkle.rs

b41ff59

Replace unwrap() calls with expect() with clear messages explaining why the operations cannot fail, following project code quality standards.

feat(ecash-migration): convert to structured logging

ff0b77f

Convert logging statements to structured format (field = value) as per project standards in the upload_spend_book function.

docs(ecash-migration): standardize terminology to 'transfer'

89a5cf1

Replace inconsistent use of 'liability transfer' with 'transfer' throughout documentation and comments for simplicity and consistency. The module name 'ecash-migration' provides sufficient context.

fix(ecash-migration-tests): only run for recent versions of fedimint

c73473c

elsirion force-pushed the 2025-11-ecash-migration branch from d04707c to c73473c Compare December 20, 2025 16:53

dpc reviewed Dec 23, 2025

View reviewed changes

dpc reviewed Dec 24, 2025

View reviewed changes

elsirion marked this pull request as draft January 5, 2026 17:14

		@@ -0,0 +1,148 @@
		//! Naive threshold signature scheme implementation using concatenated Schnorr

		@@ -0,0 +1,553 @@
		//! Merkle tree implementation for streaming data.

		@@ -1,148 +0,0 @@
		//! Naive threshold signature scheme implementation using concatenated Schnorr

Ecash migration module #7976

Are you sure you want to change the base?

Ecash migration module #7976

Uh oh!

Conversation

elsirion commented Nov 21, 2025

Uh oh!

elsirion commented Dec 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

elsirion commented Dec 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpc Dec 23, 2025 •

edited

Loading

dpc Dec 23, 2025 •

edited

Loading

dpc Dec 23, 2025 •

edited

Loading

dpc Dec 23, 2025 •

edited

Loading

dpc Dec 23, 2025 •

edited

Loading

dpc Dec 24, 2025 •

edited

Loading

dpc Dec 24, 2025 •

edited

Loading

dpc Dec 24, 2025 •

edited

Loading