[ty] Support cancellation and retry in the server

[MichaReiser]

Summary

This PR implements the following releated features:

• Support re-trying requests after they failed due to a salsa Cancellation (some content in the db was modified)
• Support for the LSP cancel request: Don't retry requests that were cancelled. Don't run background requests that were cancelled before they run on the background thread

This required significant refactoring of the server's request/notification/respond model because tracking the pending requests requires access to Session, which we don't have in background tasks.

This PR:

• It removes Requester, Responder and Notifier and unifies them under a single Client API. I found them more confusing than helpful, and I don't see any reason why we should restrict background tasks from sending requests.
• It removes ClientSender: This is mostly a fallout of the above.
• Split IOThreads out of Connection: This removes the entire complexity around weak references because scoping rules now ensure that Connection (and its senders) are dropped before we join the thread.
• Remove the global MESSENGER and instead require callers to explicitly pass a Client. This helped find show_message call sites where MESSENGER wasn't initialized. It also removes global state, which is always good
• Responses are sent to the main loop and not directly to the client. This is required to mark the request as complete (or omit the response if the request was cancelled), and it is only possible with a mut reference to Session that background tasks don't have. This approach is the same as r-a's.
• Notifications and requests are still sent directly to the client to reduce load on the main loop. We could change that if we want but I decided against it for now.

Closes astral-sh/ty#75

Test Plan

• I added a timeout in the background task scheduling and verified that requests weren't processed that were cancelled by the client
• I added a timeout to completions (that now has retry enabled) and verified that the request is re-executed after a salsa cancellation
• I tested that a panic on the main loop still shows a message box
• I tested that a panic in a background task shows a message box
• I tested that the server correctly restarts when issuing the restart server command

[github-actions]

mypy_primer results

No ecosystem changes detected ✅

[sharkdp]

Thank you very much. This all looks very reasonable, but I can not review this as deeply as it might be necessary. I'm not (yet) familiar with the LSP server code.

[dhruvmanila]

This is great! It simplifies a bunch of stuff.

A few questions / notes going through the diff:

• The spec mentions that cancellation would make it possible to return partial results. I don't think that's supported right now and I don't even think it's required or necessary now or in the near future until we've completed most of the capabilities. I don't know if partial results is going to be useful and if so how much.
• Does the current implementation only "records" that this request has been cancelled and not actually cancel the running thread?

[MichaReiser]

The spec mentions that cancellation would make it possible to return partial results. I don't think that's supported right now and I don't even think it's required or necessary now or in the near future until we've completed most of the capabilities. I don't know if partial results is going to be useful and if so how much.

That's correct. The current implementation always omits the entire result. Partial results would complicate things a bit because

The server can't send the cancelled response immediately if the request is already in progress. We also need to pass the CancellationToken to the request handler so that it can decide itself what it wants to do if it gets cancelled. I'm not sure how well that would work overall because cancellations are mainly issued after changes and changes result in a write to the db -> any db access unwinds with a Salsa::Cancelled payload. But definetely something worth exploring in the future

Does the current implementation only "records" that this request has been cancelled and not actually cancel the running thread?

This is correct. However, I don't think it should matter in most cases because the main reason for a client to send a cancellation is when some content changed, which results in mutating the database. Salsa implements mutation by unwinding all other db clones when they are accessed the next time. What this means in practice is that salsa cancels in-flight requests. However, we could pass the CancellationToken to the request handler if we have some computationally intensive logic that doesn't perform any database access.