-
Notifications
You must be signed in to change notification settings - Fork 0
Labels
bugSomething isn't workingSomething isn't working
Description
Improve --help output and "run binary only" behavior; add coretrace-testkit coverage
Description
The compiler CLI needs two UX/behavior improvements:
- The
--helpoutput should be clearer, more complete, and consistent with the actual behavior/options. - Running the produced binary by itself (i.e., “launching the binary only”) should be improved so the workflow is predictable and ergonomic.
In addition, we should add automated tests for these behaviors using the coretrace-testkit framework to prevent regressions.
Motivation / Problem
--helpis part of the public interface: users rely on it to discover options and understand correct invocation patterns.- The “compile then run” workflow (or “run only the resulting binary”) is common, and current behavior is not sufficiently polished/explicit.
- Lack of test coverage means CLI regressions can slip in unnoticed.
Scope
1) --help improvements
- Ensure
--helpaccurately documents:- All supported flags/options (including aliases if any).
- Default values and expected formats (paths, enums, booleans).
- Examples of common invocations (compile, compile+run if supported, run-only workflow).
- Improve readability:
- Group options by category (inputs, outputs, instrumentation/analysis, verbosity/logging, advanced).
- Consistent wording and ordering.
- Make error/exit codes discoverable if relevant.
2) Improve “run binary only” behavior
- Make the “launch the binary alone” flow explicit and robust:
- Define what “run only” means in terms of inputs/outputs and which arguments are forwarded (if any).
- Ensure the behavior is consistent across platforms (macOS/Linux).
- Provide clear diagnostics if the binary does not exist / is not executable / wrong format.
Note: This issue does not assume a specific flag name. The goal is to improve the behavior and document it clearly.
3) Add tests with coretrace-testkit
Add automated tests that validate:
--helpoutput:- Command exits successfully.
- Output contains key sections/flags (sanity checks).
- No obvious missing/empty sections (baseline expectations).
- “run binary only” flow:
- A small hello-world program compiles, then the resulting binary can be launched.
- Expected stdout/stderr is produced.
- Non-zero exit is propagated correctly when the program fails (negative test).
- Clear error message when trying to run a missing output binary.
Acceptance Criteria
--helpis complete, consistent, and includes examples for typical workflows.- Running the produced binary alone is clearly defined, reliable, and user-friendly.
coretrace-testkitincludes a dedicated test module covering--helpand run-only behavior.- CI executes these tests and they pass on supported platforms.
Suggested Test Plan (coretrace-testkit)
test_help_smoke.py
- Invoke
compiler --help - Assert exit code == 0
- Assert presence of expected markers (e.g., “Usage”, “Options”, core flags)
test_run_binary_only.py
- Create a minimal C/C++ source in a temp workspace
- Compile -> verify binary exists + correct format (ELF/Mach-O)
- Run binary -> assert stdout contains expected text
- Negative cases:
- Run without binary present -> assert error + exit code
- Program that returns non-zero -> assert propagation
Additional Notes
- Keep the help output stable enough to test (avoid volatile formatting unless necessary).
- If output formatting is expected to evolve, prefer testing “must-have substrings/sections” over exact full-text matching.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working