Enable PoV reclaim in all system chains

[kianenigma]

as described in https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/guides/enable_pov_reclaim/index.html

[ggwpez]

@skunert do you think we can go ahead with this, or is something missing?

[skunert]

At this time I think our proof weight story is not as accurate as we would like it to be.

• PoV reclaim underestimates proof size paritytech/polkadot-sdk#5229: This will be mitigated by having a proper base-extrinsic weight benchmarked, which will be possible after Add overhead benchmark to frame-omni-bencher paritytech/polkadot-sdk#5891 and Balances: Arbitrary number of genesis accounts paritytech/polkadot-sdk#6040
• Then there is a more recent discovery here: Storage reclaim overshooting paritytech/polkadot-sdk#6020. This one is not really reclaim specific and also needs some changes to our benchmarking code. I think @bkchr is working on this (at least he assigned himself).

These issues lead to inaccuracies but we have safeguards in place to prevent overshooting the limits:

1. Collators are still using pov_limit / 2 as the client-side size limit
2. StorageWeightReclaim: set to node pov size if higher paritytech/polkadot-sdk#5281

However, these are workarounds while we have the proper fixes in progress. I expect the situation to get a lot better in the next releases, so if we can we should hold off a bit longer.

[kianenigma]

Thanks for the update @skunert

This is not super time critical, but we probably want it in place before the AHM is done.

@seadanda short term, I highly suggest using PoV reclaim + its workarounds in ah-next runtime that you are working on, so that we can more accurately experiment with the throughput empirically.

[seadanda]

@skunert how's the progress on this? Ideally we'd enable PoV reclaim on AH before we deploy AHM, especially for staking on a parachain. Having a quick read around it looks like the your first point can be mitigated only after the release of stable2503, is this correct?

If we're happy with the current workarounds + safeguards I'd suggest sticking with that and adding PoV reclaim to AH-kusama before the migration on Kusama (release ~end of April), and trying to benchmark the base extrinsic weight properly before the AHM on Polkadot (Q3). Does that sound reasonable?

[skunert]

Yes, that sounds reasonable to me. The main runtime-side problems were fixed with paritytech/polkadot-sdk#6140, I expect accuracy to be much higher now. Since the TransactionExtensions are benchmarked, the base extrinsic weight loses a bit of its importance IMO (but yeah better still be properly benchmarked).