CIP-0172? | Self-Usable Transaction Outputs

[carlostome]

Allow transactions to use scripts and data from their own outputs during validation.

(rendered proposal)

Notes:

• I've left a TODO in the Implementation plan, since there seem to be two viable alternatives: either implement it in Conway (via a hardfork) or make it part of the upcoming Dijkstra era.

[rphair]

@carlostome I'm marking this Triage: to introduce at the CIP meeting in about 12 hours' time (https://hackmd.io/@cip-editors/126) if we can... though I would feel strange rushing to review this without a Ledger reviewer (cc @lehins @WhatisRT) first taking a quick look to confirm this is possible: maybe also confirming that it's consistent with other Ledger design goals.

Also @colll78 @polinavino @fallen-icarus especially from your interaction with the Nested Transactions CIP I think we would benefit from your insights about this proposal: including views on use cases, efficiency, safety, and other factors that have come up before. Personally I can't yet understand how this proposal and CIP-0118 are intertwined & I hope to get a better view of that through upcoming review.

[fallen-icarus]

@carlostome Do you know of any concrete use cases for this? As a dApp developer, I've never found myself wishing for this ability.

[carlostome]

@carlostome Do you know of any concrete use cases for this? As a dApp developer, I've never found myself wishing for this ability.

Hi @fallen-icarus, in the CIP, I have outlined the use case I have in mind. That is, making sharing scripts between top- and sub- level transactions (in the context of Nested transactions) independent on the order in which they are processed within a batch. I'm happy to add more explanation and/or clarifications if you find it is not clear enough.

I do not know of other use cases at the moment.

[fallen-icarus]

... making sharing scripts between top- and sub- level transactions (in the context of Nested transactions) independent on the order in which they are processed within a batch.

This is an abstract use case. What real world product needs this ability? Do batchers need this for aggregating sub-txs? Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

[rphair]

summary notes from today's CIP meeting:

➤ Use cases: as noted above, this question also came up at the meeting. My understanding is that this demand has already been met: when we combine @fallen-icarus's enquiry about use cases with @carlostome's response that "Nested Transactions" is itself the use case, it appears that "use cases of Nested Transactions" are also generally use cases of this proposal.

• @carlostome therefore, since that statement is already in your CIP document at this time, if possible please further qualify that set of use cases by confirming which particular patterns of nested transactions would benefit (most) from this proposal.

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.

• @carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

My own list of review questions that came up are:

• Can making such a change — with any implied requirements on whether grouped transactions are "ordered" or "atomic" — break anything currently under development that would take the existing Ledger rules for granted?
• Would implementing this have any performance implications for the Ledger and/or the efficiency of regular or Nested transactions?
• @lehins @WhatisRT what about the "complexity cost" of adding this? Even if feasible to add (as asked already in this preliminary review, and as @colll78 also enquired, would it be worth the technical burden to accommodate this proposal?

Mainly pending Ledger response about feasibility we chose to leave this Unconfirmed until at least the feasibility & acceptable cost of a Ledger implementation can be established.

[carlostome]

What real world product needs this ability?

I don't know.

Do batchers need this for aggregating sub-txs?

This allows batchers to reduce duplication (and thus fees) in the case of cyclic dependencies between data and/or scripts in the top- and sub-level transactions.

Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

Because batchers cannot modify sub-txs since they are signed. Scripts are not signed but data is (via scriptIntegrityHash; see the Alonzo spec).

[carlostome]

@carlostome therefore, since that statement is already in your CIP document at this time, if possible you should further qualify that set of use cases by confirming which particular patterns of nested transactions would benefit (most) from this proposal.

I've changed the CIP to reflect the particular nature of the use case within nested transactions.

My own list of review questions that came up are:

Can making such a change — with any implied requirements on whether grouped transactions are "ordered" or "atomic" — break anything currently under development that would take the existing Ledger rules for granted?

I'm not sure what the scope of anything currently under development is, but a priori yes, in the sense that this change modifies the behaviour of some part of the Ledger rules.

Would implementing this have any performance implications for the Ledger and/or the efficiency of regular or Nested transactions?

I can't foresee any implications in terms of performance and/or efficiency. However, this change will definitely simplify the implementation of data and script sharing in Nested transactions.

[carlostome]

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.

@carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

[WhatisRT]

what about the "complexity cost" of adding this? Even if feasible to add (as asked already in #1135 (comment), and as @colll78 also enquired, would it be worth the technical burden to accommodate this proposal?

This feature was originally motivated by a simplification of the logic of the specification. So any implementation that implements the logic in a similar manner should also see a simplification in their code. Roughly speaking, checking that everything is ordered correctly is extra work, and this CIP recognises that this extra work is unnecessary and not doing it gives some extra flexibility to DApp authors.

[rphair]

Can making such a change... break anything currently under development that would take the existing Ledger rules for granted?

I'm not sure what the scope of anything currently under development is, but a priori yes, in the sense that this change modifies the behaviour of some part of the Ledger rules.

apologies @carlostome: I didn't mean to introduce a potentially infinite scope. 😅 What I meant more practically was (if you might know of some answers): What common design patterns in Cardano development would depend upon the unmodified Ledger rules and therefore be affected by this CIP?

The editors (among others) would just be trying to estimate the trade impact of making this change, and so anything you (or other reviewers) can provide here would be helpful: especially if you add these scenarios to the CIP itself.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

@colll78 can you please document here this example as you described it at yesterday's CIP meeting — a typical project launch, beginning with a token mint, with this CIP avoiding waiting periods for prior transactions to be verified before continuing to each new step — which was considered an advantage and use case for this CIP? I think we need your precision here so the author knows what to do with that suggestion & perhaps include it as a use case.

---

@carlostome please note that my writing above is mainly feedback from a CIP meeting yesterday (held biweekly) in which I put your proposal on the agenda for introduction. We can't rely on those present at the meeting to record their meeting statements back on GitHub... so during or after the meetings one of the CIP editors will generally try to document it back here. We do our best to record these statements but often can't perfectly reproduce the same point of view or technical understanding. You'd be very welcome at the meeting yourself & please find the schedule here: https://luma.com/cardano-cip

[lehins]

I believe it should have been implemented this way right from the start when reference scripts where introduced as a feature, because I can't see a single reason why it shouldn't have been done that way, especially since implementation for this is very simple. Luckily we should be able to retroactively enable this feature for PlutusV1-V3 scripts as well, without much effort, since witnesses are not supplied in the script context, so scripts cannot know for sure whether script was provided as a witness or as a reference script.

In my opinion, this CIP is simply an optimization. All it does is allows a user to use a script in the same transaction that adds it to the UTxO for sub-subsequent transaction to reuse as reference script. In a common case, it does not really make something that was previously impossible possible, except when a script that a person is trying to use is pretty big in size. It only makes some certain use cases cheaper, simpler for users and reduce amount of data placed on chain. This point is quite simple to prove: any transaction that could benefit out of using a script that was supplied in one of the outputs that it is creating, could always also add it to the witness set. So, all this CIP will allow is to do is accommodate those cases without including such script in a transaction twice, which is also why scripts that are so large in size that two copies of it would not fit in a transaction, could really benefit from this CIP.

Some benefits from this CIP that come to mind:

• From @colll78 the example of minting a token, at least as far as I imagine it, maybe @colll78 has a different use case in mind. Today in order to be able mint and burn a token, while using a reference script you require at least 3 transactions: first transaction adds a script to UTxO, while subsequent two that mint and burn use that script. With this CIP the same transaction that does the minting also could add the script to UTxO for the transaction that does the burning to use. So less fees and less contention for ephemeral tokens.
• A similar use case as for token minting will be applicable for CIP-160 - Protected Adddresses. Because then the script that needs to be executed for output creation can contain that said script, so later when it is being spent there will no longer going to be a need to even specify where the script is coming from
• Simplifies usage of reference scripts in Nested Transactions. Also by its nature Nested Transactions will have many transactions in it with a size limit of a single transaction, so anything we can do to reduce binary overhead of a transaction will be beneficial for Nested Transaction especially
• Unlike scripts that are supplied through reference inputs these scripts provided by the output do not carry the same dangers of not being priced by the regular fee calculatoin. Therefore, those scripts will not need to participate in the same costing function as reference scripts, since they will already be priced through the size of transaction, similarly to how scripts that are used through a witnesses are priced.

Best thing I like about this CIP is that implementation of this feature is very simple and, as far as I can see it, there are only benefits and no drawbacks.

Big 👍 from me.

[lehins]

This is an abstract use case. What real world product needs this ability? Do batchers need this for aggregating sub-txs? Why wouldn't they just have the sub-txs omit the required script/data from the sub-tx so they can just add it to the top-level?

I can try and make it concrete. Here is an example without this CIP:

• Some sub-transaction creators start requiring the same script in topLevelRequiredGuards, while supplying said script in the witness set, or not supplying it at all. (It can be any script type, it doesn't have to be a guard script, I've used guards for the sake of example)
• Most likely aggregator (batch creator) has two priorities in mind: quickest transaction acceptance on chain and lowering the cost. Without this CIP however, aggregator can only optimize for one of those two:
  1. include that script in a witness set to be able to use it right away and submit another transaction later (or if the script fits also create an output int the same transaction) adding the script for further reuse.
  2. create another transaction, wait for it to be accepted on chain and then start using that script.

With this CIP, top-level transaction builder will have this ability to add a script to UTxO that any of the sub-transactions might need for immediate and sub-subsequent use. So, IMHO, this CIP is especially useful for the top-level transaction builder.

[rphair]

@carlostome @lehins the elaboration of Ledger impact in #1135 (comment) abundantly addresses the only blocker we had at our last CIP meeting — uncertainty over whether this could be implemented easily enough in the Ledger — and so I'd like to promptly confirm this as a CIP candidate: as we would have if we'd been sure of this on Tuesday's meeting.

I've added this back on Review for the next CIP meeting (https://hackmd.io/@cip-editors/127), where we could do that: but if both @Ryun1 and @perturbing agree here in the meantime we'd have unanimity & can re-tag this as Confirmed, assign a CIP number, and start tagging other devs for a deep (technical) and broad (ecosystem) review.

[Ryun1]

@rphair
happy to have a CIP number applied 💪

[colll78]

➤ Thanks mainly to @colll78, we highlighted a substantial benefit for this proposal (by avoiding an "ordering dependency") in that protocol creators wouldn't have to wait for the effects of something like a token minting to be processed by the chain in a previous transaction (whose Tx would be far too big to include as another Tx input) before submitting further transactions to define that protocol.
@carlostome - likewise, please consider adding this scenario (waiting for large transactions to complete first, if I've understood correctly) to your Motivation if you think it's relevant.

I don't fully understand this scenario and how it could benefit from script sharing. In particular I don't understand the part about submitting further transactions to define the protocol after minting the token.

Currently to deploy just about any DeFi protocol or dApp on Cardano requires multiple transactions, even for an extremely simple dApp that only has a single script.

The minimal flow for a stateful dApp at the moment is roughly:
Transaction 1: Deploys reference script UTxOs
Transaction 2: Include the reference script UTxOs that were produced in the previous transaction as reference inputs, and use them to mint the relevant state tokens / dynamic parameters UTxO NFT.

With this proposal you could deploy such applications in a single transaction that produces the reference script UTxOs (ie. that contain the minting policy of the state tokens) and mint the state tokens in the same transaction (ie. create a DEX pool in the same tx where you publish the DEX pool script).

[rphair]

Since all editors who left this Unconfirmed at last week's CIP meeting have now supported making this a candidate, we're promoting it without waiting for the next meeting. @carlostome please rename the containing directory to CIP-0172 and updated the "rendered" link in your original comment. 🎉

Also please consider adding those couple of things to the Motivation and Rationale to resolve the comments in #1135 (comment) & #1135 (comment) based on @Ryun1's & other editorial feedback.

Though this proposal is of general developer interest and we would welcome comments from anyone, I'm tagging some further reviewers with a good eye for overall design patterns — especially contributors to the Nested Transactions discussion & roundtable — to review the CIP and maybe help us plan how to introduce this to the dev community:

@perturbing @Quantumplation @polinavino @ch1bo @sandtreader @AndrewWestberg @michele-nuzzi @disassembler @MicroProofs

[ch1bo]

Gave a look because I got notified through Github and I could not really follow the motivation for this.

[ch1bo]

I don't follow the argument of re-using inlined data.

If a transaction wants to spend an output that carries inlined data, that data would already be in the utxo set and thus the creating tx paid for it. For "normal" datum use, there would only be a datumhash on the utxo and the spending transaction provides the preimage in the witness set. This data resolves across any spent inputs that are parameterized by the same data (hash). I don't know how this evolves for nested transactions but I would expect it to allow re-use of hashed data. For other validation purposes, there is no datum in the context.

So, what exactly motivates the sketched change of getDatum in the agda spec?

[lehins]

I do agree with this point, since datums are not reusable from reference inputs either. This change alone would lead to strange semantic where datum from outputs created can be reused throughout a transaction, while ones specified in inputs and reference inputs cannot. If we are to make such a change it would require its own CIP. However, I vaguely remember discussing this point long time ago with some folks that actually write DApps and their argument against was that it is very rare that a script requires the same data and it is usually fairly small anyways.
I think it would be worthwhile analyzing the chain and see whether there could actually be some space savings if we were to add such feature of inline datums being shared throughout a transaction.

[rphair]

There were enough people getting enthusiastically behind this proposal shortly after it was submitted — and repeatedly confirmed — that I think it needs to ride that momentum out into whatever discussion circles would lead to its implementation & then its usage by Cardano projects & protocol developers.

@carlostome the only remaining review point #1135 (comment) would be nice to have an answer — in the CIP, if possible — but not necessary since I think the general case of this feature's usability has already been established without making such a detailed study.

Generally we would not merge this without other editors' opportunity to review, which often happens in anticipation of a Last Check discussion so I've tagged it as such for the next CIP meeting: https://hackmd.io/@cip-editors/129

[lehins]

Aside from inclusion of inline datums in this proposal and few other minor points this CIP looks great.

[rphair]

@carlostome I'm taking this back from the optimistically scheduled Last Check since we still have TODO in the Path to Active (requested to be updated here: #1135 (comment))) and other review points that the editors would generally need to you or the contributor to mark "resolved" with at least a quick comment about how it's been fixed.

If the Implementation Plan is still TODO based on uncertainty about the era of implementation, we can leave it in this state while you determine that: just please comment here when the document is complete with this information (or if reviewers can help with that decision) & when it's ready for another Ledger + editor review.

[fallen-icarus]

@kylixafonso Flash loans are already possible if dApps just compose inside the same tx like I'm pushing for with the DeFi Kernel philosophy. You can only spend the UTxO with the liquidity if you return it in the same tx + a fee. No ledger changes are needed; just different dApp architectures.

[kylixafonso]

@kylixafonso Flash loans are already possible if dApps just compose inside the same tx like I'm pushing for with the DeFi Kernel philosophy. You can only spend the UTxO with the liquidity if you return it in the same tx + a fee. No ledger changes are needed; just different dApp architectures.

I think you're conflating two fundamentally different concepts.

What you're describing is transaction-level atomic composition: liquidity can be accessed and must be returned within the same transaction. That's certainly a useful pattern, but it's not equivalent to what most people mean by a flash loan, which derives its properties from broader state transitions and block-level execution context.

More importantly, I don't think sequencer-based/shared-state architectures and fragmented UTxO architectures should be treated as mutually exclusive or ideologically opposed design choices. Different applications have different requirements, and the ecosystem benefits from supporting both approaches.

In practice, many of the lending and liquidity protocols that have achieved meaningful scale across the industry - including the dominant lending protocols on Cardano - Liqwid - rely on pooled liquidity and shared-state semantics. Supporting those designs requires access to richer block-level context and coordination mechanisms, rather than restricting everything to strict transaction atomicity.

So the question isn't whether transaction-level composition is possible - it clearly is. The question is whether the platform should also support the broader class of architectures that many large-scale DeFi systems depend on. I would strongly argue that it should.

[rphair]

@carlostome @lehins (as the two principal review & response parties) although this was stalled I can see that it has passed through two rounds of review while:

• attracting some support for the pathological cases that it would solve (better explained by @colll78 above & confirming what was noted in the prior CIP meeting);
• the unique confirmation from @lehins that "it should have originally been designed to work this way" (paraphrased from memory);
• answering questions from community experts (e.g. @ch1bo) who might not originally have understood the purpose for this but appear to have all questions answered here.

The one remaining item that pulled this back from Last Check before was an uncertainty about which hard fork this would be included in (as documented in the OP #1135 (comment)).

• I don't see why this should be a problem since we've had many CIPs over the years which defined future changes to the Ledger but only indicated that "a hard fork" would be required without divining any hard fork names or scope in advance.
• Therefore I've installed a simple Path to Active that resolves that TODO item — giving us a Path to Active consistent with the Nested Transactions work scope & timeline — as explained already in this comment & change.

I think this puts us safely back at Last Check but please @lehins feel free to go over this again: especially if there are any Ledger considerations that should be added to the Path to Active to make it plausible.

• I would hope to merge this with no further reservations but please post if so & we'll take it off the agenda for now: https://hackmd.io/@cip-editors/141

Note I'll be submitting some commits below to correct the validation problems but these will just be formatting & not substantial changes.