Bump executorlib from 1.8.0 to 1.8.2

[dependabot]

Bumps executorlib from 1.8.0 to 1.8.2.

Release notes

Sourced from executorlib's releases.

executorlib-1.8.2

What's Changed

• Fix TestClusterExecutor get_item_from_future by @​jan-janssen in pyiron/executorlib#910
• [Fix] SingleNodeExecutor wait for dependencies on shutdown by @​jan-janssen in pyiron/executorlib#911
• [Fix] Only add processes which exist in process_dict by @​jan-janssen in pyiron/executorlib#913
• [Feature] Implement get_future_from_file() by @​jan-janssen in pyiron/executorlib#915
• [Feature] Implement terminate_task_in_cache() by @​jan-janssen in pyiron/executorlib#914
• [Fix] Accept benchmarks when block_allocation is faster than the processpoolexecutor by @​jan-janssen in pyiron/executorlib#917
• [Feature] ClusterExecutor: Store file errors in output files by @​jan-janssen in pyiron/executorlib#918
• [Feature] Add refresh rate for ClusterExecutors by @​jan-janssen in pyiron/executorlib#919
• [Fix] Clean up shutdown routine for ClusterExecutors by @​jan-janssen in pyiron/executorlib#920

Full Changelog: pyiron/executorlib@executorlib-1.8.1...executorlib-1.8.2

executorlib-1.8.1

What's Changed

• [pre-commit.ci] pre-commit autoupdate by @​pre-commit-ci[bot] in pyiron/executorlib#904
• [Fix] Set Pythonpath for TestClusterExecutor by @​jan-janssen in pyiron/executorlib#907
• Fix environment for interactive executors by @​jan-janssen in pyiron/executorlib#908
• Fix: Implement one _refresh_memory_dict() function by @​jan-janssen in pyiron/executorlib#902
• Feature: Cancel processes of cancelled futures by @​jan-janssen in pyiron/executorlib#903

Full Changelog: pyiron/executorlib@executorlib-1.8.0...executorlib-1.8.1

Commits

• 7e9bf9b [Fix] Clean up shutdown routine for ClusterExecutors (#920)
• d450829 [Feature] Add refresh rate for ClusterExecutors (#919)
• 6f3cc53 [Feature] ClusterExecutor: Store file errors in output files (#918)
• 3fd4fb3 [Bug]: Accept benchmarks when block_allocation is faster than the processpool...
• 0dfb64c [Feature] Implement terminate_task_in_cache() (#914)
• 7e4a14c [Feature] Implement get_future_from_file() (#915)
• f3af366 [Fix] Only add processes which exist in process_dict (#913)
• eb420fd [Fix] SingleNodeExecutor wait for dependencies on shutdown (#911)
• 5f31a73 Fix TestClusterExecutor get_item_from_future (#910)
• 4a96b1c Feature: Cancel processes of cancelled futures (#903)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[github-actions]

👈 Launch a binder notebook on branch pyiron/pyiron_workflow/dependabot/pip/executorlib-1.8.2

[liamhuber]

Both the discovery and interuption tests, which ran fine in executorlib-1.8.0 now fail on 1.8.2 with

exception calling callback for <Future at 0x7fc6b9d44350 state=finished returned int>
Traceback (most recent call last):
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/pyiron_workflow/mixin/run.py", line 353, in _finish_run
    raise e
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/pyiron_workflow/mixin/run.py", line 347, in _finish_run
    unique_executor.shutdown(wait=False)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/executorlib/executor/base.py", line 162, in shutdown
    self._task_scheduler.shutdown(wait=wait, cancel_futures=cancel_futures)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/executorlib/task_scheduler/base.py", line 205, in shutdown
    self._process.join()
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/threading.py", line 1146, in join
    raise RuntimeError("cannot join current thread")

In both cases we start a job with a NodeSlurmExecutor, which is just a SlurmClusterExecutor that gets a special "cache_key" and "cache_directory" in its resources dict. Then we kill the job hard with os._exit(0).

In both cases, after a brief pause we load the workflow. It's in its running state and we've recovered our executor construction instrucitons. For the "interrupt" case we then immediately rerun to try to connect to the still running slurm jobs, then sleep for long enough to let it finish; for the "discover" case we sleep for a long time first to let the slurm jobs finish, then re-run to discover the finished serialized files. In both cases we finally check the result by running the callback on the future. That's the part that triggers the error.

Nothing has changed on the pyiron_workflow side. @jan-janssen, I've tried reading through the recent issues and PRs on executorlib, but I don't see what is causing this changed behaviour. Is the error clear to you? Do I maybe need to provide an explicit value for "wait" or "cancel_futures" in my resource dict now?

If it's not obvious to you, I'm happy to dig in further myself. I just don't want to burn a bunch of time if it's something clear to you in five minutes.

[dependabot]

Superseded by #809.