-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Rust: regenerate MaD files using DCA #19674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
31d1604
Bulk model generator: switch from json to yml configuration files
900a3b0
MaD generator: only pick up last database on comparison DCAs
d5c16d6
MaD generator: reformat
31954fa
MaD generator: make bulk generator executable
fbd5058
MaD generator: move bulk generation config files one directory up
4f47ee2
MaD: make bulk generator DCA strategy download DBs in parallel
ee7eb86
MaD: make bulk generator cleanup downloaded DBs
530b990
MaD generator: some final minor tweaks
f4bbef9
Rust: switch to DCA strategy for MaD bulk generation
ec77eb3
Rust: regenerate MaD models
6162cf5
Rust: accept test changes
e1eb1f6
Rust: address review
d6d13b9
MaD generator: use `decompress` terminology instead of `extract`
redsun82 e6056f9
Update rust/ql/test/query-tests/security/CWE-770/main.rs
redsun82 bcfc009
Merge branch 'main' into redsun82/mad
redsun82 ecc35e5
MaD generator: run formatter
redsun82 ca99add
MaD generator: address review
redsun82 0d03699
Merge branch 'main' into redsun82/mad
redsun82 4ac4e44
Rust: accept test changes
redsun82 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,45 +1,18 @@ | ||
| strategy: repo | ||
| strategy: dca | ||
| language: rust | ||
| destination: rust/ql/lib/ext/generated | ||
| extractor_options: | ||
| - cargo_features='*' | ||
| targets: | ||
| - git_repo: https://github.com/rust-lang/libc | ||
| git_tag: 0.2.172 | ||
| name: libc | ||
| - git_repo: https://github.com/rust-lang/log | ||
| git_tag: 0.4.27 | ||
| name: log | ||
| - git_repo: https://github.com/BurntSushi/memchr | ||
| git_tag: 2.7.4 | ||
| name: memchr | ||
| - git_repo: https://github.com/matklad/once_cell | ||
| git_tag: v1.21.3 | ||
| name: once_cell | ||
| - git_repo: https://github.com/rust-random/rand | ||
| git_tag: 0.9.1 | ||
| name: rand | ||
| - git_repo: https://github.com/servo/rust-smallvec | ||
| git_tag: v1.15.0 | ||
| name: smallvec | ||
| - git_repo: https://github.com/serde-rs/serde | ||
| git_tag: v1.0.219 | ||
| name: serde | ||
| - git_repo: https://github.com/tokio-rs/tokio | ||
| git_tag: tokio-1.45.0 | ||
| name: tokio | ||
| - git_repo: https://github.com/seanmonstar/reqwest | ||
| git_tag: v0.12.15 | ||
| name: reqwest | ||
| - git_repo: https://github.com/SergioBenitez/Rocket | ||
| git_tag: v0.5.1 | ||
| name: rocket | ||
| - git_repo: https://github.com/actix/actix-web | ||
| git_tag: web-v4.11.0 | ||
| name: actix-web | ||
| - git_repo: https://github.com/hyperium/hyper | ||
| git_tag: v1.6.0 | ||
| name: hyper | ||
| - git_repo: https://github.com/clap-rs/clap | ||
| git_tag: v4.5.38 | ||
| name: clap | ||
| - name: rust | ||
| - name: libc | ||
| - name: log | ||
| - name: memchr | ||
| - name: once_cell | ||
| - name: rand | ||
| - name: smallvec | ||
| - name: serde | ||
| - name: tokio | ||
| - name: reqwest | ||
| - name: rocket | ||
| - name: actix-web | ||
| - name: hyper | ||
| - name: clap | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a nice simple list, once everything is merged and stable I'll add a bunch more targets to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one thing to keep in mind is that at the moment this list needs to be topologically ordered with respect to dependencies (so later additions should depend on earlier ones and not the other way around). Possibly worth a comment here, now that this is yaml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, just so you know, you can tweak what gets generated with any of
(all are true by default)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the expected use cases for those three options?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really know, but you can ask Mathias once he's back from his PTO, two of them are used for the C++ generated models
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My guess is there are certain libraries that produce a lot of inaccurate models of one type but not the others, and this gives us some additional control. @MathiasVP ? (no rush, not blocking this PR)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason is simply that C++ doesn't yet autogenerate sources and sinks (for a couple of reasons, but mainly because I didn't bother to set that properly up yet). The MaD generator script (which this script invokes under the hood) already provides these hooks to configure which kinds of models are generated, so I just lifted those hooks to this script in #19627