CIP-0170 | Add digest computation specification

[Kammerlo]

Fixes #1249
Specifies how the d digest of an ATTEST transaction is computed, which the CIP previously left open (no canonical serialisation, see #1249):

• If the transaction carries application metadata under another label, d MUST be the CESR Blake3-256 digest of the CBOR bytes of the metadatum value at that label, byte-identical to the transaction's auxiliary data - not a JSON or re-serialised form.
• If no application metadata is present, the data referred to by d is defined by the use case, which must specify how a verifier obtains it and what encoding is digested.
• Verifiers MUST NOT recompute d from indexer JSON (db-sync jsonb reorders keys at write time); the spec lists sources for the raw bytes (tx CBOR, db-sync tx_metadata.bytes, Blockfrost /metadata/cbor).
• Adds a self-contained test vector (positive and a reordered negative) and a rationale paragraph.

This is a clarification only: no change to the metadata format or CDDL, v.v stays 1.0. It matches the existing reference implementations (e.g. Reeve's KeriService: blake3_256(CBOR(map))).

---

Rendered newly added section (most of the update): Digest computation

[styamanda]

Thanks @Kammerlo — this is more thorough than I expected, and it resolves both things I asked about: it pins down which bytes (the metadatum value only, not the label/value pair or the whole map), and it states plainly that JSON representations cannot be used.

I verified the test vector independently and every value in it checks out:

Vector 1 CBOR	decodes to exactly the JSON shown, in construction order ✅
Vector 2 CBOR	same content, keys length-then-bytewise ✅
Blake3-256 of vector 1	ea4c2099…d75f391 — matches ✅
CESR qb64	EOpMIJmAaiP4cZgmDkg8rVtl8YU4dDYf_gxrK2sNdfOR — matches ✅
Vector 2 qb64	EIswjSN1u14y36RGf6TKm-z65R0rMMLjQHnjXofImTmZ — matches ✅

Computed with signify-ts' Diger, independently of whatever produced them. Having a vector where the wrong answer is also given is a nice touch — an implementer who lands on EIswjSN1… immediately knows what they did.

Two suggestions, one substantive.

1. Warn about the ordering between anchoring and building. The text says to digest "the bytes that will be (or were) submitted", which is right, but understates a trap. d has to be anchored in the KEL before the transaction is built, and the KEL is append-only — so if the builder's encoding differs at all from what was digested, the attestation is permanently unverifiable and a sequence number has been burned on a bad anchor. Most builders take a JSON-ish object and encode it themselves, so the issuer does not directly control the emitted bytes; schemaVersion: 1 as 01 versus 1901 is enough to break it.

Something like:

Because the KEL anchor is created before the transaction exists and cannot be retracted, implementations SHOULD extract the encoded metadatum bytes from the built transaction and confirm they digest to d before submitting.

That is a cheap check and it converts a permanent failure into a caught one.

2. Minor: the worked example is now less self-consistent than the test vector. The ATTEST example at label 1447 still carries "{{someApplicationMetadata}}" alongside a concrete digest, so a reader cannot check it — while the new §Digest computation gives fully reproducible values. Using the test vector's payload in the worked example would make the whole document verifiable end to end.

Neither blocks merging from my side. We will implement against this text — our current implementation digests a canonicalised JSON form, which was a workaround for exactly this gap — and I will report back on whether the spec was sufficient to implement from without rediscovering anything.

[styamanda]

Implemented against this text and it holds up — here is the report I promised.

Result: a conformant attestation on preprod, verifying against the new rule: 046c0ce9…. d is the Blake3-256 digest of the CBOR encoding of the metadatum value at the application label, read back from the transaction's raw CBOR. Implementation: https://github.com/Wistkey/acespeak-cip0170-spike (src/cardano/cbor.ts).

The spec text was sufficient to implement from. I did not have to guess at anything, and the test vector caught my encoder before the chain did.

One thing I would add, because it nearly bit me. The text says to digest the bytes that will be submitted, which is correct — but "encode the value to CBOR" is not a single well-defined operation in practice, and the choice is easy to get wrong invisibly. Concretely, in cardano-multiplatform-lib:

• TransactionMetadatum.from_json(...) with the detailed tagged schema preserves key order
• encode_json_str_to_metadatum(..., BasicConversions) sorts keys lexicographically

Both are reasonable-looking ways to turn a JSON object into a metadatum, they produce different bytes, and only one matches what the transaction builder emits. Picking the wrong one gives you a digest that no transaction will ever match — and because d is anchored in the KEL before the transaction is built, and the KEL is append-only, the mistake is already permanent by the time you find out.

That is what the SHOULD I suggested earlier is really protecting against, and I would now put it more strongly than "compare before submitting":

Implementations SHOULD NOT independently re-serialise the payload to compute d. Where the digest must be computed before the transaction exists, implementations SHOULD extract the encoded metadatum bytes from the built transaction and confirm they digest to d before submitting, since the KEL anchor cannot be retracted if they differ.

Ours now builds, extracts the metadatum back out of the built transaction, and refuses to submit unless it digests to the anchored value.

On the two suggestions from my earlier review: the anchor-ordering one is above. The other — aligning the worked example at label 1447 with the test vector so the whole document is reproducible — I still think is worth doing, but it is cosmetic and I would not hold the PR for it.

For what it's worth as regression material, there are now two superseded attestations of ours on preprod that look well-formed and can never verify: f690640a… (digested a JSON serialisation) and 9f84c95d… (digested a canonicalised JSON form). Both are left on chain deliberately and are useful for testing that a verifier rejects what it should.

Thanks for turning this round so quickly — happy to see it merge.

[rphair]

@Kammerlo @styamanda this is scheduled to confirm as a CIP update (Triage) at the next CIP meeting — https://hackmd.io/@cip-editors/142 — although if both of you agree that the changes are finalised I'll approve it myself & promote this to Last Check at that meeting.

I'll also tag the other editors for advance review as soon as that happens. Of course any pending or even hypothetical implementors of CIP-0170 are also encouraged to comment here in the meantime.

[Kammerlo]

@styamanda Thanks that's a good point! I added a sentence to clarify that.
If you are good with it now, then @rphair can promote it.

[rphair]

yes @Kammerlo since that appears to resolve the last reservation: pending any further testing as this develops & @styamanda submitting a further update if & when needed. This will be ready for Last Check at the next CIP meeting.

@styamanda if there is anything further you think could/should change to save you & perhaps other implementors the trouble later, please feel free to suggest further changes in the meantime.

[Kammerlo]

Thanks a lot @rphair From my point of view this update was important and a really good extension from @styamanda
I wrote the update according to how we implemented it anyway so turning the unkown into something explicit.
From my point of view it is good and matches what I had in mind anyway.