Asana Toolkit #381
Asana Toolkit #381
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
This is awesome & great coverage of the API.
One thing to think about: From what the evals are telling me, language models don't seem to fully grasp that they can provide multiple workspace_ids, or multiple tags in a single request. How can the annotations be improved to clear up this confusion?
| break | ||
| if project_name_lower in names_lower: | ||
| matches.append(project) | ||
| names_lower.remove(project_name_lower) |
There was a problem hiding this comment.
asana lets you create more than one project with the same name. This line is telling me that in the case of two projects named the same, only one of them would be returned by the tool.
There was a problem hiding this comment.
That is correct. I thought about that scenario. If the user does have 2+ projects with the same name, that's going to be an issue even in the GUI. Do you think we should raise an error saying it's not possible to resolve which project it is referring to?
| ) -> Annotated[dict[str, Any], "The created tag."]: | ||
| """Create a tag in Asana""" | ||
| if not 1 <= len(name) <= 100: | ||
| raise ToolExecutionError("Tag name must be between 1 and 100 characters long.") |
There was a problem hiding this comment.
this feels like a preventable error for > 100 chars case. just use the first 100 chars?
There was a problem hiding this comment.
In general, I think raising an error is better than truncating when the user is creating a data point to avoid issues with silently unfulfilled expectations.
Asana docs says:
Name of the tag. This is generally a short sentence fragment that fits on a line in the UI for maximum readability. However, it can be longer.
My limit to 100 chars was arbitrary. I tried creating a tag with 5,000 chars and they still allow. I don't know if there's a hard limit, but it's ridiculous to allow a large tag name like this...
| "Match tasks that started on OR BEFORE this date. Format: YYYY-MM-DD. Ex: '2025-01-01' " | ||
| "Defaults to None (searches tasks started on any date or without a start date).", | ||
| ] = None, | ||
| completed: Annotated[ |
There was a problem hiding this comment.
there's no way to get intermediate status like 'in progress', 'in review', etc?
|
This is an area for improvement. I tried to make some tools more ergonomic. Like the Even 4o is a bit flaky when taking advantage of this ergonomy. Sometimes, it'll invoke So far, I haven't found a way to make them consistent through the argument annotation or the function docstring. Did any ideas pop to your mind that we could try? |
Having them as tools was confusign the LLMs when calling create_task, update_task, or search_tasks. Instead of passing a tag name as arg to create_task, for instance, the LLM was thinking it was supposed to call search_tags_by_name first, which is unnecessary.
|
The |
Sorry this first version of the toolkit ended up getting too big. Task is the most relevant object, but it's 'entangled' with everything else in Asana. I realized the toolkit wouldn't be of much value if we didn't have some basic tools related to projects, workspaces, teams, users, and tags as well.