[Backend Tester] Add flow-specific skipped tests, skip argmin/max on Core ML due to native crashes, float16/64 on Vulkan

[GregoryComer]

Argmin and argmax are crashing on Core ML. Native crashes are hard to recover from. In the medium term, I'd like to look at using pytest-xdist, which, by nature of process isolation, can recover. But for now, I'm adding a way to skip tests on specific backend flows to allow running the remaining tests. I've also skipped Vulkan float16 and float64 tests, as they crash when using swiftshader (used in the CI for software execution of the Vulkan compute shaders). This should be re-visited in the future.

There is also a little technical debt in the code duplication between operator and model tests. This will also be cleaned up shortly.

[GregoryComer]

Stack from ghstack (oldest at bottom):

• -> [Backend Tester] Add flow-specific skipped tests, skip argmin/max on Core ML due to native crashes, float16/64 on Vulkan #13992
• [Backend Tester] Add error message field to CSV report #13991
• [Backend Tester] Skip inf and nan operator tests #13990
• [Backend Tester] Skip in-place activation tests due to lack of support in ET #13989

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13992

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 15 New Failures, 1 Cancelled Job, 1 Unrelated Failure

As of commit f30b0bf with merge base 41c299f ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  At least one of the pre-conditions you specified did not hold
• Lint / lintrunner / linux-job (gh)
  >>> Lint for backends/test/suite/operators/__init__.py:
• nightly / backend-test-linux (qnn_16a16w, models) / linux-job (gh)
  RuntimeError: Command docker exec -t e84efcf12cef4e04863f0040d92598b44181d3307c92ba7dc5548fc68d1fffcd /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a16w, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t 31f09062e319667c245a0f7813b6777a49926c901ee4350d0e9f3f71fbcbf197 /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a4w_block, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t 9933425f1ca8855cb1ec342a2c121dad2236a7f8bf2468003863f19b6916f165 /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a4w, models) / linux-job (gh)
  RuntimeError: Command docker exec -t 29bd61e9500f5b8f89f9221058698d66be711babcb104ec5f8b5610be0b9730f /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a4w, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t 5d574a593b5700c6967bdde9073c737bb055fe6496970231c2cd733f3ec73355 /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a8w, models) / linux-job (gh)
  RuntimeError: Command docker exec -t 17dcba4f68e1c5a54d4c4ea69b29ae545c46fc1b1ee82d1dc60bb33071aaa05b /exec failed with exit code 1
• nightly / backend-test-linux (qnn_16a8w, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t ab0d5a46ee235a9a8fc41613f8dde256fdbaacea89a4d524f892e645c99d7e5f /exec failed with exit code 1
• nightly / backend-test-linux (qnn_8a8w, models) / linux-job (gh)
  RuntimeError: Command docker exec -t 9f2e5f2bc3c9d7f259f79eef4073fea3c9bcb70419250a3bb1d564ca809ea61d /exec failed with exit code 1
• nightly / backend-test-linux (qnn_8a8w, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t fbb666fb53222854d0c46b64f261809b4f7c145bc52312820ac58d0a2af0f7e1 /exec failed with exit code 1
• nightly / backend-test-linux (qnn, models) / linux-job (gh)
  RuntimeError: Command docker exec -t eac564bfa6030e653777decd4c9cb91f8ccdc947d4e5192a7cf7cd3cd660f684 /exec failed with exit code 1
• nightly / backend-test-linux (qnn, operators) / linux-job (gh)
  RuntimeError: Command docker exec -t 6fa1c1264cd9932d9a59702418f7ab7be738e5062bcc280d19877213a31fa33a /exec failed with exit code 1
• pull / android / run-emulator (gh)
  The process '/usr/bin/sh' failed with exit code 255
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 7aa69edd3c9738f90c8ca9126a2705385e704f9cc1e416fe1dcde3144817f21b /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• nightly / backend-test-macos (coreml_static_int8, operators) / macos-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• nightly / backend-test-linux (qnn_16a4w_block, models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.