Introduce Clock.add_system_event_trigger(...) to include logcontext by default

[MadLittleMods]

Introduce Clock.add_system_event_trigger(...) to wrap system event callback code in a logcontext, ensuring we can identify which server generated the logs.

Background:

synapse/docs/log_contexts.md

Lines 71 to 81 in 9cc4001

	Ideally, nothing from the Synapse homeserver would be logged against the `sentinel`
	logcontext as we want to know which server the logs came from. In practice, this is not
	always the case yet especially outside of request handling.
	Global things outside of Synapse (e.g. Twisted reactor code) should run in the
	`sentinel` logcontext. It's only when it calls into application code that a logcontext
	gets activated. This means the reactor should be started in the `sentinel` logcontext,
	and any time an awaitable yields control back to the reactor, it should reset the
	logcontext to be the `sentinel` logcontext. This is important to avoid leaking the
	current logcontext to the reactor (which would then get picked up and associated with
	the next thing the reactor does).

Also adds a lint to prefer Clock.add_system_event_trigger(...) over reactor.addSystemEventTrigger(...)

Part of #18905

Dev notes

Adding logcontext to looping_call and call_later in #18907

Add logcontext to call_when_running in #18944

Mypy lint pattern came from #18828

Todo

• Wait for Introduce Clock.call_when_running(...) to include logcontext by default #18944 to merge

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Code style is correct (run the linters)

[reivilibre]

nice approach

[MadLittleMods]

Thanks for the review @reivilibre 🐠