Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update testing guide with code coverage for fuzzing #1183

Merged
merged 6 commits into from
Jan 19, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/build/guides/testing/code-coverage.mdx
Original file line number Diff line number Diff line change
@@ -49,5 +49,12 @@ cargo llvm-cov test --lcov --output-path=lcov.info

Load the `lcov.info` file into your IDE using it's coverage feature. In VSCode this can be done by installing the [Coverage Gutters] extension and executing the `Coverage Gutters: Watch` command.

:::info

Measuring code coverage in fuzz tests requires different tooling. See [Fuzzing].

:::

[Coverage Gutters]: https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
[Mutation Testing]: mutation-testing.mdx
[Fuzzing]: fuzzing.mdx
43 changes: 43 additions & 0 deletions docs/build/guides/testing/fuzzing.mdx
Original file line number Diff line number Diff line change
@@ -116,6 +116,47 @@ There is another tool for fuzzing Rust code, `cargo-afl`. See the [Rust Fuzz boo

:::

### How to Get Code Coverage of Fuzz Tests

Getting code coverage data for fuzz tests requires some different tooling than when doing the same for regular Rust tests.

1. Run the fuzz tests until it has produced a corpus, just as in step 7 above.

```
cargo +nightly fuzz run --sanitizer thread fuzz_target_1
```

2. Install the llvm-tools for the nightly compiler.

```
rustup component add --toolchain nightly llvm-tools-preview
```

3. Run the fuzz coverage command that'll execute the corpus and write coverage data to the coverage directory in the `profdata` format.

```
cargo +nightly fuzz coverage --sanitizer thread fuzz_target_1
```

4. Run the llvm-cov command to convert the profdata file to an lcov file.

```
LLVM_TOOLS_PATH=$(dirname $(find $(rustc +nightly --print sysroot) -name 'llvm-cov'))
$LLVM_TOOLS_PATH/llvm-cov export -instr-profile=fuzz/coverage/fuzz_target_1/coverage.profdata \
-format=lcov \
-object target/aarch64-apple-darwin/coverage/aarch64-apple-darwin/release/fuzz_target_1 \
--ignore-filename-regex "rustc" > lcov.info
```

Load the `lcov.info` file into your IDE using it's coverage feature. In VSCode this can be done by installing the [Coverage Gutters] extension and executing the `Coverage Gutters: Watch` command.

:::tip

To measure code coverage of regular Rust tests, see [Code Coverage].

:::


[increment example]: https://github.com/stellar/soroban-examples/blob/main/increment/src/lib.rs
[Differential Testing with Test Snapshots]: ./differential-tests-with-test-snapshots.mdx
[stellar contract fetch]: ../../../tools/developer-tools/cli/stellar-cli.mdx#stellar-contract-fetch
@@ -124,3 +165,5 @@ There is another tool for fuzzing Rust code, `cargo-afl`. See the [Rust Fuzz boo
[stellar/rs-soroban-sdk#1360]: https://github.com/stellar/rs-soroban-sdk/issues/1360
[fuzzing example]: ../../smart-contracts/example-contracts/fuzzing.mdx
[Rust Fuzz Book]: https://rust-fuzz.github.io/book
[Code Coverage]: code-coverage.mdx
[Coverage Gutters]: https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters