Skip to content

[Bug]: Pyxis environment for SWE bench is calling one srun per tool call, extremely long call time and crashes #521

Description

@leopck

Bug Description

In the original implementaton for this, #438, the pyxis env creates one srun per tool call for SWE bench, and if we run multiple SWE bench at the same time, this explodes the slurm controller and cause extremely bad slow down for SWE bench on Pyxis env.

The fix is to do a Persistent Pyxis session which tool calls can use a single session instead of every tool call is one srun.

Steps to Reproduce

  1. Run multiple SWE bench at the same time and track the step calls

Environment

Pyxis environment

Relevant Logs

Before submitting

  • I searched existing issues and found no duplicates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions