Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"/api/chat" returns a different result compared to "/api/chat/request" #485

Open
foxgem opened this issue Jan 18, 2025 · 0 comments
Open

Comments

@foxgem
Copy link

foxgem commented Jan 18, 2025

When a request was sent on the FastAPI swagger ui page, for the same message,

  • /api/chat/request could return an expected result.
  • /api/chat could not, the response was:
Image
0:""
8:[{"type": "sources", "data": {"nodes": []}}]
0:"```"
0:"```\nThought: The current language of the user is: English. I need to"
0:" use a tool to help me answer the question.\nAction: query_index"
0:"\nAction Input: {\"input\": \"standards for letters\"}\n```\n"

After checking the output on the console, I found it was actually the same as the first part of the log messages from /api/chat/request. The following was the log when calling /api/chat/request:

> Running step 449a2807-75b9-4da4-9298-603a6341a9d7. Step input: What standards for letters exist?
Thought: The current language of the user is: English. I need to use a tool to help me answer the question.
Action: query_index
Action Input: {'input': 'standards for letters'}
Observation: The standards for letters include a weight limit of 70 pounds and a combined length and girth limit of 108 inches. Lower size or weight standards may apply to mail addressed to certain APOs and FPOs, and for Department of State mail.

> Running step 305ba39f-8366-4798-aaab-1690bef5bfb9. Step input: None
Thought: I can answer without using any more tools. I'll use the user's language to answer
Answer: Standards for letters include a weight limit of 70 pounds and a combined length and girth limit of 108 inches. Lower size or weight standards may apply to mail addressed to certain APOs and FPOs, and for Department of State mail.
INFO:     127.0.0.1:55188 - "POST /api/chat/request HTTP/1.1" 200 OK
...

Looks like the streaming interface returned the earlier, not waiting for the final result of the user query?

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant