How to prevent some tool calls to be streamed while neatly saving everything in my db?

[peguerosdc]

Hi! I have developed an agent using openai-agents-js that looks like this:

const finalAgent = new Agent({ name: 'Come up with the final response to be sent to the user' });

Agent.create({
  name: 'Orchestrator',
  instructions: 'Call the tools as needed and when ready, handoff to the finalAgent.',
  tools: [tool1, tool2, tool3],
  handoffs: [finalAgent],
});

(I am not attached to using the openai's sdk, but I thought it was a simple way of explaining my goal)

And I want to integrate it with ai-sdk mostly to use the rich features of useChat. Figuring out how to code that pattern using ai-sdk is easy, but I have some particular requirements that I'm not sure how to achieve:

1. calls to tool1 and tool2 must not be streamed to the user. Instead, a generic "executing tool" message should be shown
2. calls to tool3 must be streamed to the user.
3. the output of finalAgent must be streamed to the user
4. after finalAgent is finished, the streaming stops
5. I want to save every part of the execution flow in a database

What I have found so far:

Option 1: use stream transformation

I can use the streamText function to code the agent pattern that I described above and use experimental_transformStream to filter the stream and remove the parts related to tool1 and tool2, as suggested in the documentation:

You can use the experimental_transform option to transform the stream. This is useful for e.g. filtering, changing, or smoothing the text stream.

The transformations are applied before the callbacks are invoked and the promises are resolved. If you e.g. have a transformation that changes all text to uppercase, the onFinish callback will receive the transformed text.

My concerns are:

• to filter the calls to tool3, would I have to understand, code and maintain all the Stream Protocols details?
• As stated in the docs, I wouldn't have all the parts available in onFinish, making it problematic to store them in a database as exemplified in the chats sdk, being in direct conflict with requirement 5. Why would it be problematic? Because in onFinish I can just have a dump of everything and pass it to a saveChat function without having to worry about the order of things, which could be a problem if the data is spread in different variables.

Option 2: use createUIMessageStream

I could follow the multi-step agent example in the cookbox by:

• using createUIMessageStream to create a useChat-compatible stream
• code the Orchestrator agent "manually" (i.e. without streamText, maybe with the openai sdk), make sure to stream the calls to tool3, and code finalAgent with streamText to merge the streams and make it available to the user

My concerns are similar to option 1:

• to manually stream the calls to tool3, would I have to understand, code and maintain all the Stream Protocols details? This seems to be a lot of work and error-prone.
• I wouldn't have all the parts available in onFinish, making it problematic to store everything in a database.

Any suggestions?

I was hoping someone could give me some suggestions on how to achieve this or feedback on my requirements. Some thoughts that I'm having:

• I am hesitant of losing some parts of the stream in onFinish (or by not using streamText) because from the chatbot example, it seems that it's important to save everything in the DB to load it back easily to display the chat history to the user. Does this make sense?
• I want to store everything in the DB to later to analytics of how to improve the agent, the tools, etc. Should I try to store the execution flow independently of the parts?
• Understanding the Stream Protocols maybe is not that difficult, just that there's not a lot of documentation of how to do it. Should I try to understand it or is it not worth the effort? Is it subject to too much change?
• ai-sdk is the right tool for this?

[github-actions[bot]]

This discussion was automatically locked because it has not been updated in over 30 days. If you still have questions about this topic, please ask us at community.vercel.com/ai-sdk