Skip to content

Conversation

@yuhuchua-qti
Copy link

Description

This change introduces new environment variables to make debugging failed unit tests easier by saving input/output artifacts:

  • ORT_UNIT_TEST_DUMP_ARTIFACTS: Dumps inputs, outputs, and expected outputs for test cases.
  • ORT_UNIT_TEST_ARTIFACTS_DIR: Sets the directory where artifacts are stored. Defaults to the current working directory if not provided.

Examples

Example 1: Using ORT_UNIT_TEST_DUMP_ARTIFACTS
PowerShell: $Env:ORT_UNIT_TEST_DUMP_ARTIFACTS = "1";

C:.
├───QnnCPUBackendTests_DISABLED_Gemm_Broadcast_Bias_DynamicA_StaticB_DynamicC  # TestModelAccuracy 
│   ├───CPU_EP_outputs
│   │       output.pb
│   │
│   ├───inputs
│   │       input.pb
│   │       input_token_0.pb
│   │
│   └───QNNExecutionProvider_outputs
│           output.pb
│
└───QnnHTPBackendTests_DISABLED_BinaryOp_Div4D_LargeInputs  # TestQDQModelAccuracy 
    ├───CPU_EP_F32_outputs
    │       output.pb
    │
    ├───CPU_EP_QDQ_outputs
    │       output.pb
    │
    ├───inputs
    │       input.pb
    │       input_token_5.pb
    │
    └───QNN_EP_QDQ_outputs
            output.pb

Motivation
This change introduces artifact dumping capabilities to make it
easier to inspect test data and diagnose issues quickly.

Description
Introduced the following environment variables to control artifact
dumping behavior:
- ORT_UNIT_TEST_DUMP_ARTIFACTS: Dumps inputs, outputs, and expected
                                outputs for testcases. Artifacts
				would be saved in protobuf format
				(.pb).
- ORT_UNIT_TEST_ARTIFACTS_DIR: Specifies the directory where
                               artifacts are stored. Defaults to the
			       current working directory if not
			       provided.
@yuhuchua-qti yuhuchua-qti changed the title Dump UT artifacts for debugging [QNN EP] Dump UT artifacts for debugging Nov 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant