[BUG]: Inference does not stop when clicking away from active session

[gregtakacs]

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

This had been reported before, but closed, however I could repro this issue on 1.14.1 just today using LM Studio as the model provider.

#2640

If you move from an active chat it closes the connection (this in and of itself is a dubious design decision without real benefit) however the inference does not stop on the other end and it keeps generating tokens into thin air. While this may be a nuisance on a local LLM provider it can be a costly API issue using remote frontier API tokens.

This is a very serious bug.

Are there known steps to reproduce?

set up LM Studio and use it as the model provider in AnythingLLM. Ask an agent question then click away and back. Notice that the question as well as the inference progress is gone in the chat. Go to the server side of LM Studio and observe that there is a active token process still generating tokens.

LLM Provider & Model (if applicable)

LM Studio with qwen/qwen3.6-35b-a3b

Embedder Provider & Model (if applicable)

No response

[gregtakacs]

OK, I tested with Ollama as well, same story, it keeps running (GPU load is high) inference after I clicked away.

However, I also noticed that if I let the inference finish and then go back to that chat stream the result actually gets loaded/populated. So this means that the inference is running and its results do get loaded into the chat window eventually. The issue in this case is:

1. there is no indication that this is happening (the thinking bar goes away)
2. the /exit does not work, it actually spawns a new inference on the /exit token

[gregtakacs]

Another issue is that the thread name does not change to a new name until the user clicks on another thread so there is absolutely zero indication that anything happened in the thread and the results populated back.

[LaSurga]

+1 this has been the case since at least v1.10. the 'STOP INFERENCE' signal does not work, either in this case, or when the stop button is pressed, when using LMStudio as the LLM server.

[MustafaKemal0146]

I reproduced the agent-session cancellation gap on the current master branch: closing the client WebSocket only closes the invocation record, while the active AIbitat/provider stream receives no abort signal. I am working on a focused fix that propagates disconnect/stop cancellation to the in-flight provider request and adds regression coverage. I will keep the scope limited to the reported cancellation path.