PoC: Optimize key creation

[jeltz]

This optimizes key creation in both when doing it and in the WAL replay. Plus fixes a bug when we did unnecessary work when working with temporary table.

Optimizations:

• Do not log a separate delete key entry when creating an encrypted table since we can delete the old key as part of replaying the key creating
• Replace the old key when scanning the key file instead of first deleting and then adding
• Very ugly and experimental use of fread() instead of pread() to reduce number of syscalls

[codecov-commenter]

Codecov Report

❌ Patch coverage is 94.23077% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (TDE_REL_17_STABLE@602cd73). Learn more about missing BASE report.
⚠️ Report is 1 commits behind head on TDE_REL_17_STABLE.

Additional details and impacted files

@@                 Coverage Diff                  @@
##             TDE_REL_17_STABLE     #502   +/-   ##
====================================================
  Coverage                     ?   82.24%           
====================================================
  Files                        ?       25           
  Lines                        ?     3199           
  Branches                     ?      519           
====================================================
  Hits                         ?     2631           
  Misses                       ?      457           
  Partials                     ?      111           

Components	Coverage Δ
access	83.44% <90.32%> (?)
catalog	87.61% <ø> (?)
common	77.77% <ø> (?)
encryption	72.97% <ø> (?)
keyring	73.21% <ø> (?)
src	94.15% <ø> (?)
smgr	95.90% <100.00%> (?)
transam	∅ <ø> (?)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[AndersAstrand]

I wonder if we should explore more drastic options, such as having one key file per relation, to just remove the issue of having to scan the key file completely.

It feels like it's always going to be a bottle neck when dealing with a lot of tables (ie, large key files)

[dutow]

I wonder if we should explore more drastic options, such as having one key file per relation, to just remove the issue of having to scan the key file completely.

Definitely, but later :)

[AndersAstrand]

Another idea is to always add keys to the end of the file and take the cost while reading instead. Not sure that's what we want to do though depending on what we want to optimize for.

[jeltz]

Part of this has already been merged as #529. The rest is for another day.