Code

Refactor validate feature#158

Code

Merged

PhilippvK merged 89 commits intodeveloptum-ei-eda/mlonmcu:developfrom refactor-validate-rebasedtum-ei-eda/mlonmcu:refactor-validate-rebasedCopy head branch name to clipboardSep 9, 2024

+2,505-57

Commits

Commits on Apr 11, 2024

• feature.py: RunFeatures can not enable subprocesses

  PhilippvK

  committed
  4f9ef92

  Copy full SHA for 4f9ef92

• WIP: add new features and postprocesses for validate_new

  PhilippvK

  committed
  f7bbf57

  Copy full SHA for f7bbf57

• frontends.py: fix typo in process_metadata

  PhilippvK

  committed
  8036b3f

  Copy full SHA for 8036b3f

• run.py: allow process_metadata to return artifacts

  PhilippvK

  committed
  aea3130

  Copy full SHA for aea3130

• WIP: add new features and postprocesses for validate_new

  PhilippvK

  committed
  b0fc7ed

  Copy full SHA for b0fc7ed

• fix print_top dtype

  PhilippvK

  committed
  a87b42a

  Copy full SHA for a87b42a

• add quant/dequant details to model_info

  PhilippvK

  committed
  d9a91f7

  Copy full SHA for d9a91f7

• WIP: add new features and postprocesses for validate_new

  PhilippvK

  committed
  4d4c98c

  Copy full SHA for 4d4c98c

• add assertion to SimpleFrontend

  PhilippvK

  committed
  3ffc3c6

  Copy full SHA for 3ffc3c6

• WIP: add new features and postprocesses for validate_new (tvm platform)

  PhilippvK

  committed
  fb4a217

  Copy full SHA for fb4a217

• fix print_top dtype 2

  PhilippvK

  committed
  59c89dc

  Copy full SHA for 59c89dc

• WIP: add new features and postprocesses for validate_new (validate_outputs postprocess)

  PhilippvK

  committed
  2e10d31

  Copy full SHA for 2e10d31

• WIP: add new features and postprocesses for validate_new

  PhilippvK

  committed
  bb5d710

  Copy full SHA for bb5d710

• tvmc_utils: disable print_top by default

  PhilippvK

  committed
  c8b0fab

  Copy full SHA for c8b0fab

• WIP: add new features and postprocesses for validate_new (mlif)

  PhilippvK

  committed
  d497193

  Copy full SHA for d497193

• WIP: add new features and postprocesses for validate_new (tvm)

  PhilippvK

  committed
  844020c

  Copy full SHA for 844020c

• tvmc_utils: disable print_top by default 2

  PhilippvK

  committed
  9a23c7f

  Copy full SHA for 9a23c7f

• WIP: add new features and postprocesses for validate_new (postproc atol rtol)

  PhilippvK

  committed
  119e13c

  Copy full SHA for 119e13c

• target/common.yml: Allow overriding stdout encoding for execute utility

  PhilippvK

  committed
  fec58b4

  Copy full SHA for fec58b4

• target/common.py: Add stdin_data argument (needs live=False)

  PhilippvK

  committed
  a2426f5

  Copy full SHA for a2426f5

• target.exec may return artifacts now

  PhilippvK

  committed
  3a0679b

  Copy full SHA for 3a0679b

• target.py: add utils for choosing io interfaces

  PhilippvK

  committed
  3ee0b4b

  Copy full SHA for 3ee0b4b

• Add ValidationNew.md

  PhilippvK

  committed
  10c1106

  Copy full SHA for 10c1106

• Update ValidationNew.md

  PhilippvK

  committed
  0f6c948

  Copy full SHA for 0f6c948

• Update ValidationNew.md

  PhilippvK

  committed
  06e08a7

  Copy full SHA for 06e08a7

• tflite frontend: add inference method

  PhilippvK

  committed
  6ec8090

  Copy full SHA for 6ec8090

• continue working on gen_ref_data feature

  PhilippvK

  committed
  c132768

  Copy full SHA for c132768

• tflite frontend: prints (WIP)

  PhilippvK

  committed
  603b1a5

  Copy full SHA for 603b1a5

• continue working on gen_ref_data feature 2

  PhilippvK

  committed
  87610a3

  Copy full SHA for 87610a3

• validate_outputs: set rtol=0 for testing

  PhilippvK

  committed
  c6b20aa

  Copy full SHA for c6b20aa

• tflite frontend: extract model info from tflite file if definition.yml missing

  PhilippvK

  committed
  67048ca

  Copy full SHA for 67048ca

• update md

  PhilippvK

  committed
  9553d0f

  Copy full SHA for 9553d0f

• mlif: support generated model_support files for validation feature

  PhilippvK

  committed
  6f26688

  Copy full SHA for 6f26688

• implement mlif template versioning

  PhilippvK

  committed
  969543f

  Copy full SHA for 969543f

• mlif_target: fix batch size calculation

  PhilippvK

  committed
  4e323cb

  Copy full SHA for 4e323cb

• mlif_target: cleanup interface selection code

  PhilippvK

  committed
  3c91733

  Copy full SHA for 3c91733

• handle mlif_iunvalid_size error correctly

  PhilippvK

  committed
  8971790

  Copy full SHA for 8971790

• pass user/env config to ModelHints

  PhilippvK

  committed
  1e90d0e

  Copy full SHA for 1e90d0e

• mlif: support generated model_support files for validation feature

  PhilippvK

  committed
  b25cbd9

  Copy full SHA for b25cbd9

• move ModelSupport code into separate class and file

  PhilippvK

  committed
  6a6da26

  Copy full SHA for 6a6da26

• mlif.py: add missing imports

  PhilippvK

  committed
  69665a4

  Copy full SHA for 69665a4

• update validation readme

  PhilippvK

  committed
  f642c68

  Copy full SHA for f642c68

• fix typo in mlonmcu/models/frontend.py

  PhilippvK

  committed
  77347a9

  Copy full SHA for 77347a9

• update validation readme

  PhilippvK

  committed
  7e0f9ae

  Copy full SHA for 7e0f9ae

• update validation readme

  PhilippvK

  committed
  577d0a5

  Copy full SHA for 577d0a5

• update validation readme

  PhilippvK

  committed
  676d75e

  Copy full SHA for 676d75e

• fix typo in gen_ref_data property

  PhilippvK

  committed
  09aeafd

  Copy full SHA for 09aeafd

• fixes

  PhilippvK

  committed
  725ac5d

  Copy full SHA for 725ac5d

• validate_outputs: add experimental pm1 metric

  PhilippvK

  committed
  0e839e1

  Copy full SHA for 0e839e1

• frontends: move generation of inout data to different methods

  PhilippvK

  committed
  3668400

  Copy full SHA for 3668400

• frontends: allow overriding model-specific paths

  PhilippvK

  committed
  f1c75e4

  Copy full SHA for f1c75e4

• lint code

  PhilippvK

  committed
  4345db5

  Copy full SHA for 4345db5

• move validate metrics to seperate class & file

  PhilippvK

  committed
  1a8d9eb

  Copy full SHA for 1a8d9eb

• platforms: add mlif.needs_model_support

  PhilippvK

  committed
  bc24886

  Copy full SHA for bc24886

• lint

  PhilippvK

  committed
  65233fd

  Copy full SHA for 65233fd

• fix merge artifact

  PhilippvK

  committed
  607c5c6

  Copy full SHA for 607c5c6

• lint

  PhilippvK

  committed
  5f198eb

  Copy full SHA for 5f198eb

• add missing file

  PhilippvK

  committed
  ab3017a

  Copy full SHA for ab3017a

Commits on May 23, 2024

• mlif: fixes

  PhilippvK

  committed
  77f2a40

  Copy full SHA for 77f2a40

• postprocesses: add missing registrations

  PhilippvK

  committed
  95c1dd8

  Copy full SHA for 95c1dd8

• postprocesses: fix typo

  PhilippvK

  committed
  c6b3e94

  Copy full SHA for c6b3e94

• postprocesses: improve validate_outputs

  PhilippvK

  committed
  2e61212

  Copy full SHA for 2e61212

• postprocesses: implement export_outputs

  PhilippvK

  committed
  4ec9d7f

  Copy full SHA for 4ec9d7f

• postprocess fixes

  PhilippvK

  committed
  8cc27b3

  Copy full SHA for 8cc27b3

• utils.execute: fixes for stdin_data feature

  PhilippvK

  committed
  6966e7e

  Copy full SHA for 6966e7e

Commits on May 24, 2024

• models: convert support_path, inputs_path & outputs_path to pathlib.Path

  PhilippvK

  committed
  e7f1be1

  Copy full SHA for e7f1be1

• postprocess fixes

  PhilippvK

  committed
  69c974b

  Copy full SHA for 69c974b

Commits on May 27, 2024

• frontends: add assertion msg

  PhilippvK

  committed
  d8d6269

  Copy full SHA for d8d6269

• rm print

  PhilippvK

  committed
  d040b1b

  Copy full SHA for d040b1b

• introduce new feature: GenRefLabels

  PhilippvK

  committed
  71e7280

  Copy full SHA for 71e7280

• introduce new feature: GenRefLabels 2

  PhilippvK

  committed
  c64bdc3

  Copy full SHA for c64bdc3

• frontend: handle dtype ranges

  PhilippvK

  committed
  983989e

  Copy full SHA for 983989e

• frontend: fix assertions

  PhilippvK

  committed
  9a15a32

  Copy full SHA for 9a15a32

• frontends: find handling of aww input shift

  PhilippvK

  committed
  86d7aa9

  Copy full SHA for 86d7aa9

• introduce new feature: GenRefLabels 3

  PhilippvK

  committed
  7b435d9

  Copy full SHA for 7b435d9

• introduce validate_labels postprocess

  PhilippvK

  committed
  916f38a

  Copy full SHA for 916f38a

• introduce classify metrics

  PhilippvK

  committed
  79baeda

  Copy full SHA for 79baeda

• validate_outputs: add validate_range option (on by default) 2

  PhilippvK

  committed
  852736e

  Copy full SHA for 852736e

• run: add error message if not frontend can be initialized

  PhilippvK

  committed
  65499ca

  Copy full SHA for 65499ca

Commits on May 28, 2024

• update validate_metrics

  PhilippvK

  committed
  cbff929

  Copy full SHA for cbff929

Commits on May 29, 2024

• Merge remote-tracking branch 'origin/main' into refactor-validate-rebased

  PhilippvK

  committed
  9196ede

  Copy full SHA for 9196ede

• lint code

  PhilippvK

  committed
  cbc5a15

  Copy full SHA for cbc5a15

• lint code 2

  PhilippvK

  committed
  73e945b

  Copy full SHA for 73e945b

Commits on Sep 9, 2024

• Merge branch 'develop' into refactor-validate-rebased3

  PhilippvK

  committed
  d97d338

  Copy full SHA for d97d338

• features: comment out unimplemented get_postprocesses

  PhilippvK

  committed
  97b3553

  Copy full SHA for 97b3553

• features: add docstrings to validation features

  PhilippvK

  committed
  a6a38c2

  Copy full SHA for a6a38c2

• features: cleanup comments

  PhilippvK

  committed
  40e90fe

  Copy full SHA for 40e90fe

• tvm_target: rm prints

  PhilippvK

  committed
  34e0e5d

  Copy full SHA for 34e0e5d

• lint

  PhilippvK

  committed
  d9d6a97

  Copy full SHA for d9d6a97