Issue Running Time-Series Development Task (ili) — No Outcome or Progress Logged

[ajpaezm]

Hi @guosyjlu,

First of all, thank you for releasing DS-Agent — it's an exciting project!

I'm currently working on a re-implementation focused exclusively on time-series tasks, and I'm trying to run the ili task using the following command:

python runner.py --task ili --llm-name gpt-3.5-turbo-16k --edit-script-llm-name gpt-3.5-turbo-16k

However, the agent doesn't seem to make any progress. The main_log shows repeated failures with the message:

The instruction cannot be perfectly performed by another Python programming Agent in 5 times. Please give a more simplified and feasible instruction and retry.
There are no code changes, no metrics, and the running log remains stuck at the initial state. I’ve attached a sample log here for reference.

Is there something I need to add to make this run correctly? I've added the benchmarks folder with the tasks.json accordingly on /development/MLAgentBench.

Thanks in advance for any help.

[guosyjlu]

Hi, could you please provide more details on the logs? What is the plan generated by CBR? What are the errors when executing the generated python script? Maybe you can check these details to figure out the reasons of the failure.