CIP-0176 | Serialize is_valid as the trailing element of each block transaction

[Lucsanszky]

What

Amends CIP-0176 to replace the block-level invalid_transactions index list with a trailing is_valid flag on each transaction in a block:

• block_body becomes a plain list of block_transactions
• block_transaction = [transaction_body, transaction_witness_set, auxiliary_data/ nil, bool], where the trailing bool is the is_valid flag set by the block producer

Why

1. The split between "transaction as serialized in a block" and "transaction as submitted to the mempool" plus a block-level index list proved confusing, especially with the addition of EBs (endorser blocks) in Leios, which also reference transactions.
2. Future fields that are set by the block producer rather than the transaction author (e.g. the proposed feeChangeAmount) would need the same treatment. Placing producer-set fields after all author-supplied fields gives them a natural, extensible position at the end of each block transaction.
3. The author-supplied fields form a contiguous prefix of each block transaction, cleanly separating the data that the transaction author signs off on from the data added at block production time (mirroring the previous segregated layout, where validity flags came last in the block body).
4. It removes the cross-field invariant ("every index must be strictly smaller than the number of transactions") that the index list required validators to check.

The block body hash is unaffected: it remains a single hash over the serialized block_body as a whole.

Motivated by IntersectMBO/cardano-ledger#5912.

Submitted in tandem with an amendment to CIP-0167 (#1262), which describes the mempool side of the same transaction wire-format split.

[rphair]

thanks @Lucsanszky; this & the concurrent update both look realistic and the maintenance on these CIPs is much appreciated.

@lehins (cc @carlostome @williamdemeo) the CIP editors can approve this & the concurrent update once confirmed consistent with the ledger spec, but also because of implications between Tx and block body I think we should tag reviewers & advocates of both CIP-0167 and CIP-0176 to be aware of possible impact: @ch1bo @colll78 @Quantumplation @Ryun1 @teodanciu

If neither disputed nor explicitly confirmed this would naturally get tagged Confirmed from Triage at the next CIP meeting (https://hackmd.io/@cip-editors/143) with Last Check for merge at the meeting after that.

[lehins]

LGTM

[rphair]

Given that both this & its concurrent PR are confirmed all around from the Ledger side, we only have the remaining question of possible ecosystem impact, as tagged already to a handful of experts above (sorry @fallen-icarus I didn't think of tagging you before; only now does it seem like this might affect transaction dependencies).

And since there's been no objection yet upon presentation this week, we would normally take the next CIP meeting (https://hackmd.io/@cip-editors/143) as an opportunity to find who else in Consensus (cc @perturbing), applications (cc @colll78 @Crypto2099), etc. might object to either of these CIPs.

Therefore, since that's what we would be doing at Last Check anyway, I'm approving this (based on apparent sensibility + consistency with that Ledger already plans to do) & marking it as such. That will give us another week for any objections & other editors (cc @Ryun1 @ch1bo) to point out any problems or other stakeholder that we should wait until getting a review from.

[ch1bo]

This is a breaking change, but one for the better and it greatly simplifies the block structure. Glad that we seemingly came to agreement about the per tx overhead being worth the simplicity!

[ch1bo]

Leios is aware 👍