Route Threshold Optimization does not change threshold values

[surajyadav91]

Hi All,

I'm training my route layer with 1000 samples across approximately 50 routes. However, I've noticed that even after calling rl.fit(), the thresholds remain constant at 0.5 for each route (both before and after training), resulting in no improvement in accuracy. I'm running this in local mode. I see many people have raised this but no solutions yet.

Additional questions:

1. Is it possible to train our routes on larger text inputs, such as full passages, rather than just the single-line examples shown in the documentation?
2. When we execute rl.fit(), does the LLM play any role? My understanding is that the LLM is only involved when we call function.
3. Is the semantic router simply performing cosine similarity between mean utterances embeddings and input embedding?

[jamescalam]

As far as I'm aware, this issue was resolved some time ago. I have an example here I put together just yesterday where I was optimizing thresholds (towards the end of the notebook).

It could be an input data issue, I would make sure to have plenty of (utterance, route) pairs in the data, covering every route and also including plenty of (utterance, None) pairs for examples of what utterances should not trigger any route. Here's an example dataset.

If anyone is still seeing issues please let us know by raising an issue