[Custom Transactions] Let TxBuilder set the HTLC dust limit

[tankyleo]

Let `TxBuilder` set the HTLC dust limit

Channel now sorts dust and nondust HTLCs according to the dust limit set
by `TxBuilder`.

We also let `TxBuilder` set the total endogenous fees spent on a set of
HTLC transactions via `htlc_txs_endogenous_fees_sat`.

Both of these changes abstract the weight of HTLC transactions away
from channel.

[ldk-reviews-bot]

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

[tankyleo]

We discussed earlier passing the entire list of HTLCs to TxBuilder, and letting it do the dust-vs-non-dust sorting itself. There are nonetheless multiple places in channel that are interested in knowing the exact HTLC dust limit (see get_pending_{inbound, outbound}_htlc_details, get_available_balances_for_scope).

So we prefer to add a single method that surfaces this limit to channel, and then let channel sort HTLCs depending on whether their value sits above or below that amount.

As a result, builders of custom transactions will only have a say on where the dust limit sits, and won't be able to choose arbitrary subsets for dust and non-dust HTLCs.

[carlaKC]

We discussed earlier passing the entire list of HTLCs to TxBuilder, and letting it do the dust-vs-non-dust sorting itself.

Is the main motivation for this to get all of the commitment-related logic out of channel.rs, or that we think that custom tx builders will want to specifically choose certain htlcs to be dust?

get_available_balances_for_scope

The key questions we're looking to answer seems to be "can I afford a commitment with this theoretical dust/nondust htlc"? This does seem to be something we could move into TxBuilder if we pass the full HTLC set and use a version of HTLCCandidate which just tells TxBuilder whether it is dust (we never actually use this amount other than the dust check).

Sadly here's no getting around needing to know the dust limit if we want to clamp our capacity to that value, so we'd still need to surface htlc_success_timeout_dust_limits.

get_pending_{inbound, outbound}_htlc_details

Tempting to suggest just adding an is_dust check to TxBuilder for these to make it slightly more abstract than surfacing the second stage fees, but given the above requirement to know the exact dust limit maybe we just want to leave as-is.

so tl;dr: I'd be interested in seeing what trying to pull more of the dust logic out into TxBuilder looks like, even if just gives us a clearer idea of where the trait line should be.

---

Meta note: This has got a lot of overlap with 3bb0586, so I think we should either:

1. Rebase Update fee and dust handling for zero fee channels #3884 on this PR
2. Pull the commit out into a common prefactor (I have a slight pref for this, but happy with either)

[tankyleo]

Is the main motivation for this to #3775 (comment), or that we think that custom tx builders will want to specifically choose certain htlcs to be dust?

I would say we focus on the latter for this PR, and let the former be the overarching goal :)

so tl;dr: I'd be interested in seeing what trying to pull more of the dust logic out into TxBuilder looks like, even if just gives us a clearer idea of where the trait line should be.

Let me know what you think of this new direction here. Still have some clunkiness to resolve, but that's what it's looking like right now.

• htlc_success_timeout_dust_limits is used only in get_available_balances_for_scope, and I will see if I can come up with a better method call that fits better with the "what is our available balance" question.
• Definitely take your point about surfacing second-stage tx fees, will review that part too. Right now we only need it to calculate dust exposure due to excess feerates set by a counterparty (in a channel where they are the funder), see https://delvingbitcoin.org/t/disclosure-irrevocable-fees-stealing-from-ln-using-revoked-commitment-transactions/1314

[tankyleo]

Meta note: This has got a lot of overlap with 3bb0586

Definitely let me rebase on top of your PR, you've rebased once already :)

[ldk-reviews-bot]

🔔 1st Reminder

Hey @carlaKC! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

[carlaKC]

Let me know what you think of this new direction here.

I like this approach! Definitely prefer being able to move a lot of the dust / second stage tx reasoning into the builder 👍

[ldk-reviews-bot]

🔔 2nd Reminder

Hey @carlaKC! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

[tankyleo]

I'm aware I still owe a follow-up PR from the previous PR in this project, will push that soon in a separate PR :)

[TheBlueMatt]

Not really sure I understand the point of these two methods if we have htlc_success_timeout_dust_limit as a method - if the API includes a method to say "hey, for an HTLC of type X what is the threshold where its dust", why bother with a method to say "given these HTLCs, how many are dust?", it seems the second can be calculated from the first, no? The same applies for passing the HTLC list to commit_tx_fee_sat.

Alternatively, we could drop the htlc_success_timeout_dust_limits method (if we want to not require there be some strict threshold to indicate when an HTLC is dust, though I think its a fine assumption to bake into the API, I dunno why someone would want to have some HTLCs be dust and others at the same value not be) and have a more generic "is HTLC dust on the next counterparty/local commitment transaction" call, but even there it seems like we don't need all of these methods.

[tankyleo]

Sounds good yes I initially had a version of the API that matches what you describe. We wanted to see what it would look like if we moved more logic out of channel into TxBuilder, but I agree we only really need the dust limits.

I will clean up those methods, and add TxBuilder methods for 1) the dust limit 2) the htlc endogenous fees (to calculate counterparty dust exposure).

[TheBlueMatt]

I mean don't get me wrong, I'd love to move more logic out of channel.rs into TxBuilder, that would be great, but we also don't want to have an overlapping API where we have two ways to do things over the API. Also, we want to eventually make TxBuilder public, so it would be nice to avoid adding too much complexity (in the form of a ton of different methods).

[tankyleo]

FWIW the real blocker on the more ambitious / complex API was the get_available_balances method - at the moment this method requires an exact dust limit above which all HTLCs are non-dust.

I have some code written that moves most of get_available_balances behind TxBuilder, but the complexity is not worth it, especially if we can make the assumption you describe above about the strict dust-vs-nondust threshold.

[ldk-reviews-bot]

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.02%. Comparing base (eac01d0) to head (1eb1c9e).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3921      +/-   ##
==========================================
- Coverage   89.02%   89.02%   -0.01%     
==========================================
  Files         167      167              
  Lines      121800   121763      -37     
  Branches   121800   121763      -37     
==========================================
- Hits       108434   108401      -33     
+ Misses      10953    10951       -2     
+ Partials     2413     2411       -2     

Flag	Coverage Δ
fuzzing	22.68% <97.61%> (-0.05%)	⬇️
tests	88.85% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[elnosh]

for what it can be worth it I have reviewed 1eb1c9e and it looks correct to me.

As for my understanding on the reasoning for the change, motivation seems to be to move usage of second_stage_tx_fees_sat and htlc_tx_fees_sat to TxBuilder in order to abstract away usage of weight of htlc transactions away from channel.rs.

[tankyleo]

for what it can be worth it I have reviewed 1eb1c9e and it looks correct to me.

As for my understanding on the reasoning for the change, motivation seems to be to move usage of second_stage_tx_fees_sat and htlc_tx_fees_sat to TxBuilder in order to abstract away usage of weight of htlc transactions away from channel.rs.

Thanks for reviewing ! I would go further and say we want to allow people to set arbitrary dust limits for HTLCs on commitment transactions.

[tankyleo]

@TheBlueMatt did you have a chance to look at the latest commit here ? It has the API cleanup, thank you !

[TheBlueMatt]

Discussed it offline, but I'm somewhat skeptical of a design that adds too many more methods on TxBuilder. We should consider exploring options that keep TxBuilder much smaller with more logic in larger methods (that we're willing to call and throw away some of the result of, eg relying more on something like that looks like build_commitment_stats to figure out dust exposures and whether an htlc is acceptable).