Fuzz impl v2 #3881

rcoh · 2024-10-17T14:15:33Z

Motivation and Context

This implements fuzzing for smithy-rs servers

Description

Testing

Checklist

For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "client," "server," or both in the applies_to key.
For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "aws-sdk-rust" in the applies_to key.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

github-actions · 2025-01-09T16:46:49Z

A new generated diff is ready to view.

No codegen difference in the AWS SDK
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2025-01-09T18:35:55Z

A new generated diff is ready to view.

No codegen difference in the AWS SDK
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

david-perez

Good to go but we should ensure we run the fuzzgen test in fuzzgen/src/test/kotlin/software/amazon/smithy/rust/codegen/fuzz/FuzzHarnessBuildPluginTest.kt in CI.

david-perez · 2025-01-10T09:49:15Z

...main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/CargoTomlGenerator.kt

@@ -113,6 +113,7 @@ class CargoTomlGenerator(
                "features" to cargoFeatures.toMap(),
            ).deepMergeWith(manifestCustomizations)

+        println(cargoToml)


Is this intentional or a leftover?

fuzzgen/src/main/kotlin/software/amazon/smithy/rust/codegen/fuzz/FuzzHarnessBuildPlugin.kt

david-perez · 2025-01-10T09:54:39Z

fuzzgen/src/test/kotlin/software/amazon/smithy/rust/codegen/fuzz/FuzzHarnessBuildPluginTest.kt

+            operations: [SayHello],
+            version: "1"
+        }
+        @optionalAuth


Is @optionalAuth relevant here?

david-perez · 2025-01-10T09:55:44Z

fuzzgen/src/test/kotlin/software/amazon/smithy/rust/codegen/fuzz/FuzzHarnessBuildPluginTest.kt

+import software.amazon.smithy.rust.codegen.core.util.runCommand
+import software.amazon.smithy.rust.codegen.server.smithy.testutil.serverIntegrationTest
+
+class FuzzHarnessBuildPluginTest() {


I don't think this test will get run by CI? Since this fuzzgen Gradle module is new.

yes fixed thanks

rust-runtime/aws-smithy-fuzz/README.md

david-perez · 2025-01-10T09:59:30Z

rust-runtime/aws-smithy-fuzz/README.md

+3. The fuzz driver can be used on other fuzz targets.
+
+### Setup
+First, you'll need to generate the 1 (or more) versions of a smithy-rs server to test against. The best way to do this is by using the smithy CLI. **This process is fully automated with the `aws-smithy-fuzz setup-smithy`. The following docs are in place in case you want to alter the behavior.**


Suggested change

First, you'll need to generate the 1 (or more) versions of a smithy-rs server to test against. The best way to do this is by using the smithy CLI. **This process is fully automated with the `aws-smithy-fuzz setup-smithy`. The following docs are in place in case you want to alter the behavior.**

First, you'll need to generate the 2 (or more) versions of a smithy-rs server to test against. The best way to do this is by using the smithy CLI. **This process is fully automated with the `aws-smithy-fuzz setup-smithy`. The following docs are in place in case you want to alter the behavior.**

you can fuzz against 1 — you are then fuzzing for panics instead of fuzzing for comparison

david-perez · 2025-01-10T10:03:19Z

rust-runtime/aws-smithy-fuzz/src/main.rs

+    exec(
+        Command::new("rm")
+            .arg("-r")
+            .arg(format!("{}/.m2", home_dir.display()))


Perhaps nuke just ~/.m2/repository/software/amazon/smithy/? Lest we overreach and delete important stuff.

Co-authored-by: david-perez <[email protected]>

…l-v2

github-actions · 2025-02-10T20:37:02Z

A new generated diff is ready to view.

No codegen difference in the AWS SDK
No codegen difference in the Client Test
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

aajtodd · 2025-02-12T18:29:27Z

fuzzgen/build.gradle.kts

@@ -0,0 +1,111 @@
+/*


Not a great place to comment so I'm just picking here. This is fine for now but I'd really like to see us refactor our project structure a bit and group codegen all together, something like:

codegen/ smithy-rs-server-codegen/ smithy-rs-client-codegen/ smithy-rs-shared-codegen/ ...

Example names but we also need to consider publishing and it's useful if the directory for a maven artifact matches the published name (1) easier to find and (2) that's the default anyway when publishing is to use the project name.

I'd like to do something similar for the runtime.

Seems reasonable

## Motivation and Context This implements fuzzing for smithy-rs servers ## Description  ## Testing    ## Checklist  - [ ] For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "client," "server," or both in the `applies_to` key. - [ ] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ --------- Co-authored-by: david-perez <[email protected]>

rcoh requested review from a team as code owners October 17, 2024 14:15

rcoh and others added 10 commits January 9, 2025 11:24

Add aws-smithy-fuzz utility crate

e6cf7a0

Fixes found from running on dev desktop

8b27461

Fixes for dependencies, other bug fixes

aaf239c

base64 decode automatically

e100113

lots more fixes (some more cleanup still required)

36447a8

git clone using HTTPs protocol

b3b7de5

rm -r ~/.m2 before invoking smithy build

0f8a496

Rollback changes to RustCrate

702252c

First pass at cleanups

be16fc4

more cleanups

d510c80

rcoh force-pushed the fuzz-impl-v2 branch from ebc094b to d510c80 Compare January 9, 2025 16:28

rcoh requested a review from drganjoo January 9, 2025 16:29

CI cleanups

c945a74

david-perez approved these changes Jan 10, 2025

View reviewed changes

rcoh and others added 3 commits February 10, 2025 15:16

Apply suggestions from code review

e3bf7b3

Co-authored-by: david-perez <[email protected]>

Merge branch 'main' of github.com:smithy-lang/smithy-rs into fuzz-imp…

2eafcff

…l-v2

Ensure fuzz unit tests run in CI

3bd3159

rcoh enabled auto-merge February 10, 2025 20:19

drganjoo approved these changes Feb 12, 2025

View reviewed changes

aajtodd approved these changes Feb 12, 2025

View reviewed changes

rcoh added this pull request to the merge queue Feb 12, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 12, 2025

rcoh added this pull request to the merge queue Feb 13, 2025

Merged via the queue into main with commit a9d4f1d Feb 13, 2025
44 of 45 checks passed

rcoh deleted the fuzz-impl-v2 branch February 13, 2025 13:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fuzz impl v2 #3881

Fuzz impl v2 #3881

rcoh commented Oct 17, 2024

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

david-perez left a comment

david-perez Jan 10, 2025

david-perez Jan 10, 2025

rcoh Feb 10, 2025

david-perez Jan 10, 2025

rcoh Feb 10, 2025

david-perez Jan 10, 2025

rcoh Feb 10, 2025

david-perez Jan 10, 2025

github-actions bot commented Feb 10, 2025

aajtodd Feb 12, 2025

rcoh Feb 13, 2025

	First, you'll need to generate the 1 (or more) versions of a smithy-rs server to test against. The best way to do this is by using the smithy CLI. This process is fully automated with the `aws-smithy-fuzz setup-smithy`. The following docs are in place in case you want to alter the behavior.
	First, you'll need to generate the 2 (or more) versions of a smithy-rs server to test against. The best way to do this is by using the smithy CLI. This process is fully automated with the `aws-smithy-fuzz setup-smithy`. The following docs are in place in case you want to alter the behavior.

Fuzz impl v2 #3881

Fuzz impl v2 #3881

Conversation

rcoh commented Oct 17, 2024

Motivation and Context

Description

Testing

Checklist

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

david-perez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment