replace loggingcontexts with contextvars

[matrixbot]

This issue has been migrated from #10342.

---

logcontexts are super confusing, and easy to mess up.

We should experiment with replacing the thread-local variable we currently use with a contextvar, which Twisted has first-class support for (twisted/twisted#1192), which could potentially save everyone who ever touches Synapse code a great deal of pain.

[MadLittleMods]

Another spot that needs ContextVar support -> Rust create_deferred:

synapse/rust/src/http_client.rs

Lines 247 to 251 in c68c5dd

	/// Creates a twisted deferred from the given future, spawning the task on the
	/// tokio runtime.
	///
	/// Does not handle deferred cancellation or contextvars.
	fn create_deferred<'py, F, O>(

Possible way to solve:

One thing I did when I did the pyo3-twisted bridge POC was to store and restore the contextvars during the future execution. If we switched to storing the logging context in contextvars, it would solve the logcontext being lost, but maybe not the resource usage metering. You can still probably take inspiration from that?

-- @sandhose, #18903 (comment)

[MadLittleMods]

I was under the impression that Twisted supported ContextVar's but now I'm not sure. All of the issues mentioned in matrix-org/synapse#10342 are resolved but there are other things in the Twisted tracker:

Unresolved issues:

• Support contextvars in Deferred twisted/twisted#9807
  • [#9807] contextvars support for Deferred twisted/twisted#1262
• Support contextvars in DelayedCall twisted/twisted#9824

Resolved issues:

• Support Contextvars in coroutines (inlineCallbacks/ensureDeferred) twisted/twisted#9719
  • [#9719] Support contextvars in coroutines twisted/twisted#1192
• Contextvars support does not support .reset twisted/twisted#10301
  • 10301 Fix support for contextvar.reset twisted/twisted#1690