Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create wasmtime::Config from toml #9811

Merged
merged 2 commits into from
Jan 27, 2025

Conversation

ludfjig
Copy link
Contributor

@ludfjig ludfjig commented Dec 13, 2024

This PR makes it possible to create a wasmtime::Config from a toml file and is a first step toward resolving #8784. Overall, I think the path forward is as follows:

  1. This PR: Enables loading wasmtime::Config from a toml file, by making CommonOptions derive Deserialize. For example, opt_level is deserialized from values 0, 1, 2, "s" (same as cli flags) instead of "None", "Speed", "SpeedAndSize". Same applies to regalloc_algorithm, compiler and collector. Feedback appreciated on this.
  2. Combine cache-config toml with this new toml.
  3. Have one source of truth for cli flags and config options.

Feedback greatly appreciated, especially regarding how to deal with nn_graph, as well config_var and keyvalue_in_memory_data, which are all marked serde(skip) for now

@ludfjig ludfjig requested a review from a team as a code owner December 13, 2024 00:15
@ludfjig ludfjig requested review from alexcrichton and removed request for a team December 13, 2024 00:15
@ludfjig ludfjig marked this pull request as draft December 13, 2024 00:16
@ludfjig ludfjig force-pushed the ludfjig/fileconfig branch 3 times, most recently from 23b1984 to 5427eeb Compare December 13, 2024 00:39
@github-actions github-actions bot added wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime labels Dec 13, 2024
Copy link

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

  • If you added a new Config method, you wrote extensive documentation for
    it.

    Our documentation should be of the following form:

    Short, simple summary sentence.
    
    More details. These details can be multiple paragraphs. There should be
    information about not just the method, but its parameters and results as
    well.
    
    Is this method fallible? If so, when can it return an error?
    
    Can this method panic? If so, when does it panic?
    
    # Example
    
    Optional example here.
    
  • If you added a new Config method, or modified an existing one, you
    ensured that this configuration is exercised by the fuzz targets.

    For example, if you expose a new strategy for allocating the next instance
    slot inside the pooling allocator, you should ensure that at least one of our
    fuzz targets exercises that new strategy.

    Often, all that is required of you is to ensure that there is a knob for this
    configuration option in wasmtime_fuzzing::Config (or one
    of its nested structs).

    Rarely, this may require authoring a new fuzz target to specifically test this
    configuration. See our docs on fuzzing for more details.

  • If you are enabling a configuration option by default, make sure that it
    has been fuzzed for at least two weeks before turning it on by default.


To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I'm liking how this is shaping up pretty cleanly so far. Would you be up for integrating this into a CLI flag itself as well? For example wasmtime --config cfg.toml or something like that

crates/cli-flags/src/lib.rs Outdated Show resolved Hide resolved
crates/cli-flags/src/lib.rs Outdated Show resolved Hide resolved
@ludfjig
Copy link
Contributor Author

ludfjig commented Dec 13, 2024

Thanks for this! I'm liking how this is shaping up pretty cleanly so far. Would you be up for integrating this into a CLI flag itself as well? For example wasmtime --config cfg.toml or something like that

Thanks for looking at this. I added a --config flag. Any suggestions on how to best test this?

@alexcrichton
Copy link
Member

For tests I'd recommend tests/all/cli_tests.rs or somewhere around there for making a tempdir with a config file

@alexcrichton
Copy link
Member

This is looking good to me, was there something else you wanted to address before landing though?

@ludfjig
Copy link
Contributor Author

ludfjig commented Jan 22, 2025

Sorry for the delay @alexcrichton. I was wondering if I need to update any documentation, and if the current tests are sufficient. If not, I am ready for merge/reviews.

@ludfjig ludfjig marked this pull request as ready for review January 22, 2025 20:35
@ludfjig ludfjig requested a review from a team as a code owner January 22, 2025 20:35
@ludfjig ludfjig requested review from pchickey and removed request for a team January 22, 2025 20:35
Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to land, so I'm going to go ahead and approve-and-merge.

If you'd like to help write some docs I suspect that https://github.com/bytecodealliance/wasmtime/blob/main/docs/cli-options.md might be a good location to put this perhaps?

@alexcrichton alexcrichton added this pull request to the merge queue Jan 22, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 22, 2025
@ludfjig ludfjig force-pushed the ludfjig/fileconfig branch from 5bc538f to 485bb10 Compare January 27, 2025 18:48
@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 27, 2025
@ludfjig ludfjig force-pushed the ludfjig/fileconfig branch from 1a0e73a to 94627d2 Compare January 27, 2025 20:09
@ludfjig
Copy link
Contributor Author

ludfjig commented Jan 27, 2025

@alexcrichton I had some problems getting the test suite with the i686 target to pass, but it should hopefully be fixed now by using wasm-max-stack=0 which is explicitly disallowed and will cause an error.

@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 27, 2025
@alexcrichton
Copy link
Member

Let's see if that's spurious...

@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Jan 27, 2025
@alexcrichton
Copy link
Member

Ah sorry it looks like #10123 landed which causes conflicts with this. I'm happy to help out with resolving conflicts if needed.

@ludfjig ludfjig force-pushed the ludfjig/fileconfig branch from 94627d2 to 37faba3 Compare January 27, 2025 23:13
@ludfjig
Copy link
Contributor Author

ludfjig commented Jan 27, 2025

@alexcrichton no worries. The only question I have is about the fmt::Display impl for CommonOptions. If configured using a --config <path>, then the output will potentially be incorrect, since it will look like all options were provided on cli instead of from a toml. Is this acceptable?

@alexcrichton
Copy link
Member

I think that's reasonable yeah, the Display implementation here is sort of best-effort so printing out --config $path for when the field is Some I think is good enough

@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
Merged via the queue into bytecodealliance:main with commit 2a5c141 Jan 27, 2025
39 checks passed
alexcrichton added a commit to alexcrichton/wasmtime that referenced this pull request Jan 27, 2025
Remove `derive(Deserialize, Serialize)` for some configuration-based
enums to ideally give us more flexibility in the future as to what these
are exactly called. I'm not sure of any internal users of these except
for one use due to bytecodealliance#9811 which can be switched to using an option-based
parser to ensure the same syntax is accepted via `--config` and on the
CLI with a flag.
github-merge-queue bot pushed a commit that referenced this pull request Jan 28, 2025
* Cut down on serde requirements in `wasmtime` crate

Remove `derive(Deserialize, Serialize)` for some configuration-based
enums to ideally give us more flexibility in the future as to what these
are exactly called. I'm not sure of any internal users of these except
for one use due to #9811 which can be switched to using an option-based
parser to ensure the same syntax is accepted via `--config` and on the
CLI with a flag.

* Remove unused imports

* Fix option parsing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants