Code

Remote Execution Support#163

Code

Draft

PhilippvK wants to merge 84 commits intodeveloptum-ei-eda/mlonmcu:developfrom feature-process-pool-newtum-ei-eda/mlonmcu:feature-process-pool-newCopy head branch name to clipboard

+2,200-406

Commits

Commits on Mar 26, 2024

• Introduce RunInitializer and RunResult

  PhilippvK

  committed
  7f78397

  Copy full SHA for 7f78397

• drop run.session

  PhilippvK

  committed
  6c0ecc4

  Copy full SHA for 6c0ecc4

• add run.has_target()

  PhilippvK

  committed
  e52b9e8

  Copy full SHA for e52b9e8

• drop run.session 2

  PhilippvK

  committed
  e59f893

  Copy full SHA for e59f893

• fix mlonmcu/cli/common.py: runs_per_stage not detected on CMDLINE

  PhilippvK

  committed
  69e8e2f

  Copy full SHA for 69e8e2f

• add run.has_target()

  PhilippvK

  committed
  9e08e44

  Copy full SHA for 9e08e44

• Context: add MlonMcuContextMinimal

  PhilippvK

  committed
  802c666

  Copy full SHA for 802c666

• Introduce RunInitializer and RunResult

  PhilippvK

  committed
  fae52a6

  Copy full SHA for fae52a6

• Run: remove unsused self.result

  PhilippvK

  committed
  4ba7dea

  Copy full SHA for 4ba7dea

• Run: add missing self.dir = None

  PhilippvK

  committed
  9b4808f

  Copy full SHA for 9b4808f

• Introduce RunInitializer and RunResult 3

  PhilippvK

  committed
  d4abd6e

  Copy full SHA for d4abd6e

• Session: experiment with process_pool feature to deal with GIL

  PhilippvK

  committed
  693b5c9

  Copy full SHA for 693b5c9

Commits on Mar 27, 2024

• fixes

  PhilippvK

  committed
  c44fe66

  Copy full SHA for c44fe66

• fix typo

  PhilippvK

  committed
  9c94f14

  Copy full SHA for 9c94f14

Commits on Mar 30, 2024

• etiss: skip get_metrics script if trace_memory is false, and only update RAM metrics

  PhilippvK

  committed
  e72c816

  Copy full SHA for e72c816

• WIP: add workaround for run dir with unknown session

  PhilippvK

  committed
  080840c

  Copy full SHA for 080840c

Commits on May 2, 2024

• session: yield futures once they are completed

  PhilippvK

  committed
  d7f5cf3

  Copy full SHA for d7f5cf3

• session: update pbar in _join_workers

  PhilippvK

  committed
  48c132c

  Copy full SHA for 48c132c

Commits on May 4, 2024

• session: move pbar utilities to progress.py

  PhilippvK

  committed
  3d1568f

  Copy full SHA for 3d1568f

Commits on May 7, 2024

• session.py: move lots of code to schedule.py

  PhilippvK

  committed
  001fbcb

  Copy full SHA for 001fbcb

• mlonmcuscheduler fixes

  PhilippvK

  committed
  5543f66

  Copy full SHA for 5543f66

Commits on May 12, 2024

• schedule.py: add comments

  PhilippvK

  committed
  26a8959

  Copy full SHA for 26a8959

• Merge remote-tracking branch 'origin/main' into feature-as-completed

  PhilippvK

  committed
  460ded5

  Copy full SHA for 460ded5

• lint

  PhilippvK

  committed
  c796801

  Copy full SHA for c796801

Commits on May 13, 2024

• Merge branch 'feature-as-completed' into feature-process-pool-new

  PhilippvK

  committed
  0783680

  Copy full SHA for 0783680

• sessionscheduler fixes

  PhilippvK

  committed
  bd1caf9

  Copy full SHA for bd1caf9

• sessionscheduler fixes

  PhilippvK

  committed
  5d77ddf

  Copy full SHA for 5d77ddf

• begin ArchivedRun impl

  PhilippvK

  committed
  9ce8c74

  Copy full SHA for 9ce8c74

• update RunInitializer impl

  PhilippvK

  committed
  39c78e9

  Copy full SHA for 39c78e9

• revert init_directory workaround

  PhilippvK

  committed
  c31aa3e

  Copy full SHA for c31aa3e

• add run.initializer() method (untested)

  PhilippvK

  committed
  34ea7f4

  Copy full SHA for 34ea7f4

• schedule.py: cleanup imports

  PhilippvK

  committed
  0e49923

  Copy full SHA for 0e49923

• update scheduler.postprocess()

  PhilippvK

  committed
  c80fb9c

  Copy full SHA for c80fb9c

• introduce session.use_init_stage

  PhilippvK

  committed
  f33ee6c

  Copy full SHA for f33ee6c

• sessionscheduler fixes

  PhilippvK

  committed
  e8ab479

  Copy full SHA for e8ab479

Commits on May 17, 2024

• artifacts: use IntFlag instead of Enum for formats

  PhilippvK

  committed
  3d244f0

  Copy full SHA for 3d244f0

• platforms: add missing returns to init_directory

  PhilippvK

  committed
  6d8a460

  Copy full SHA for 6d8a460

• run: add save and cleanup methods for Run & RunInitializer

  PhilippvK

  committed
  95fc537

  Copy full SHA for 95fc537

• run.init_directory: pass runs_dir instead of session

  PhilippvK

  committed
  5489883

  Copy full SHA for 5489883

• add_model: add better error handling

  PhilippvK

  committed
  a648370

  Copy full SHA for a648370

• run: add save and cleanup methods for Run & RunInitializer

  PhilippvK

  committed
  98f0b74

  Copy full SHA for 98f0b74

• scheduler: add error msg for unsupported session postprocesses

  PhilippvK

  committed
  ea0ae6d

  Copy full SHA for ea0ae6d

• store run results in scheduler

  PhilippvK

  committed
  c290081

  Copy full SHA for c290081

• session: expose executor as cfg

  PhilippvK

  committed
  9d3fd1d

  Copy full SHA for 9d3fd1d

• refactor session.get_reports to use runresults instead of runs

  PhilippvK

  committed
  eb1c44d

  Copy full SHA for eb1c44d

• session: cleanup imports

  PhilippvK

  committed
  2fb3243

  Copy full SHA for 2fb3243

• session: add assertion for empty session

  PhilippvK

  committed
  ef81706

  Copy full SHA for ef81706

• cli: allow _ as dummy model name

  PhilippvK

  committed
  f11209b

  Copy full SHA for f11209b

Commits on May 19, 2024

• cli: implement --initializer arg

  PhilippvK

  committed
  d3ccdef

  Copy full SHA for d3ccdef

• cli: implement --noop arg

  PhilippvK

  committed
  6d0fc43

  Copy full SHA for 6d0fc43

• implement session.shuffle

  PhilippvK

  committed
  84822b2

  Copy full SHA for 84822b2

• scheduler: pass clean and save option as args

  PhilippvK

  committed
  97fe937

  Copy full SHA for 97fe937

• scheduler: improve handling of runinitializers and used stages

  PhilippvK

  committed
  88928be

  Copy full SHA for 88928be

• cli: implement --initializer arg 2

  PhilippvK

  committed
  afac9ba

  Copy full SHA for afac9ba

• add missing import

  PhilippvK

  committed
  79828a5

  Copy full SHA for 79828a5

• lint code

  PhilippvK

  committed
  3a69b9e

  Copy full SHA for 3a69b9e

• session: implement batching

  PhilippvK

  committed
  21d976d

  Copy full SHA for 21d976d

Commits on May 20, 2024

• cli: fix

  PhilippvK

  committed
  0ea803b

  Copy full SHA for 0ea803b

• scheduler: implement cmdline executor

  PhilippvK

  committed
  515e29e

  Copy full SHA for 515e29e

• scheduler: fixes

  PhilippvK

  committed
  9d48190

  Copy full SHA for 9d48190

Commits on May 21, 2024

• Refactor executors into classes

  PhilippvK

  committed
  d3251d5

  Copy full SHA for d3251d5

Commits on May 22, 2024

• add mlonmcu_eval.sh script

  PhilippvK

  committed
  e1cd791

  Copy full SHA for e1cd791

• add missing imports

  PhilippvK

  committed
  58cc23f

  Copy full SHA for 58cc23f

• fix print_report handling in cmdline

  PhilippvK

  committed
  2543184

  Copy full SHA for 2543184

• update mlonmcu_eval.sh script

  PhilippvK

  committed
  18f55c0

  Copy full SHA for 18f55c0

• update mlonmcu_eval.sh script

  PhilippvK

  committed
  d695c86

  Copy full SHA for d695c86

• wip: add rpc scripts

  PhilippvK

  committed
  ec311db

  Copy full SHA for ec311db

• stoire session results in class

  PhilippvK

  committed
  e50a493

  Copy full SHA for e50a493

• implement RPCSessionExecutor

  PhilippvK

  committed
  ed3d537

  Copy full SHA for ed3d537

• implement RPCSessionExecutor 2

  PhilippvK

  committed
  f4b7698

  Copy full SHA for f4b7698

• finish rpc implementation

  PhilippvK

  committed
  666f405

  Copy full SHA for 666f405

• update mlonmcu_eval.sh script

  PhilippvK

  committed
  65f247c

  Copy full SHA for 65f247c

• comment out prints

  PhilippvK

  committed
  dbec826

  Copy full SHA for dbec826

Commits on May 23, 2024

• cleanup fix

  PhilippvK

  committed
  5df6e7f

  Copy full SHA for 5df6e7f

• update mlonmcu_eval.sh script

  PhilippvK

  committed
  674d6f6

  Copy full SHA for 674d6f6

Commits on May 24, 2024

• update mlonmcu_eval.sh

  PhilippvK

  committed
  facec1f

  Copy full SHA for facec1f

• update mlonmcu_eval.sh

  PhilippvK

  committed
  5d4285e

  Copy full SHA for 5d4285e

• add eval_outs/ directory

  PhilippvK

  committed
  904960c

  Copy full SHA for 904960c

• update mlonmcu_eval.sh

  PhilippvK

  committed
  8dead62

  Copy full SHA for 8dead62

Commits on May 25, 2024

• run: fix handling of missing sub artifacts

  PhilippvK

  committed
  e284efe

  Copy full SHA for e284efe

Commits on May 29, 2024

• drop default environment cfg from class

  PhilippvK

  committed
  bd3250e

  Copy full SHA for bd3250e

• context: update create_session to allow custom dir

  PhilippvK

  committed
  69f1fc2

  Copy full SHA for 69f1fc2

Commits on Jun 7, 2024

• move eval outs from repo

  PhilippvK

  committed
  b6b0358

  Copy full SHA for b6b0358

Commits on Jun 19, 2024

• update mlonmcu eval script

  PhilippvK

  committed
  525adf8

  Copy full SHA for 525adf8