Skip to content

Corpus refactoring #3159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

Corpus refactoring #3159

wants to merge 14 commits into from
+3,729 −3,037

Conversation

rmalmain
Copy link
Member

No description provided.

tokatoka
tokatoka reviewed Apr 14, 2025
View reviewed changes
domenukk
domenukk reviewed Apr 15, 2025
View reviewed changes
libafl/Cargo.toml
], default-features = false } # A faster hashmap, nostd compatible
num-traits = { workspace = true, default-features = false }
serde = { workspace = true, features = ["alloc"] } # serialization lib
serde = { workspace = true, features = ["alloc", "rc"] } # serialization lib
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They warn against adding rc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://serde.rs/feature-flags.html#rc

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they just say it will clone the inner data, i don't think it's a problem for us

@tokatoka tokatoka added this to the LibAFL 1.0 milestone Apr 28, 2025
@domenukk
Copy link
Member

domenukk commented May 27, 2025 •
edited
Loading

One big thing we may need to address for corpus at some point is that we now keep all metadata in memory at all times.
Maybe metadata should (optionally?) be treated and cached like part of the testcase, instead

@tokatoka tokatoka mentioned this pull request Jun 26, 2025
Merged
@rmalmain rmalmain mentioned this pull request Aug 22, 2025
Open
@rmalmain
Copy link
Member Author

One big thing we may need to address for corpus at some point is that we now keep all metadata in memory at all times. Maybe metadata should (optionally?) be treated and cached like part of the testcase, instead

it is partially done this way in the patch.
but the metadata is attached to the testcase for now, so if the testcase gets flushed from the cache, the metadata will be flushed as well.
i don't know if we should keep all the metadata in memory all the time, idk if the size of metadata can explode as well

@rmalmain
create a cache cell.
e0899f4 
rename testcase metadata cell trait.
cache dirty cell writeback implementation.
implemented cache policies.
@rmalmain
better cache policy
64deb68
@rmalmain
add input to append_metadata.
2799aaa 
continue adapting new corpus stuff for more code.
add generic add function to corpus / store.
@rmalmain
possibility to disable corpus entry
5ed2c4e
@rmalmain
implemented disable everywhere
600d162
@rmalmain
fmt
7b8745b
@rmalmain
introduce collection.rs, containing all the standard corpus
286a236
@rmalmain rmalmain requested a review from tokatoka September 19, 2025 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@domenukk domenukk domenukk left review comments

@tokatoka tokatoka Awaiting requested review from tokatoka

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
LibAFL 1.0
Development

Successfully merging this pull request may close these issues.
3 participants
@rmalmain @domenukk @tokatoka