Describe the current behaviour
Currently, in our llm call endpoint we just have one endpoint where we only send the response to the given callback URL, so if a user does not have a callback url that they could input into this endpoint then they have no way of getting the LLM response.
Describe the enhancement you'd like
We can introduce a polling endpoint which polls the llm call db for llm response , and users that don't have callback url to give, they can use this endpoint to get the LLM Response. Before we implement this though, we need to research if having both these endpoint would be contradictory pattern.
Describe the current behaviour
Currently, in our llm call endpoint we just have one endpoint where we only send the response to the given callback URL, so if a user does not have a callback url that they could input into this endpoint then they have no way of getting the LLM response.
Describe the enhancement you'd like
We can introduce a polling endpoint which polls the llm call db for llm response , and users that don't have callback url to give, they can use this endpoint to get the LLM Response. Before we implement this though, we need to research if having both these endpoint would be contradictory pattern.