lcms benchmark -- seed profile seems invalid

[thuanpv]

Hi,

I am working on a few more input models for the input-structure aware fuzzer AFLSmart and I figure out that the single seed for little-cms library seems invalid. I read the code and understand that the test driver expects an ICC profile as its input. However, the sample seed doesn't adhere to the ICC profile specification -- the header is missing if I am not wrong (according to this http://www.color.org/icc32.pdf).

It seems that it is not very uncommon to see invalid seeds or empty seed in FuzzBench's benchmarks. See issue #463

I understand that we should evaluate fuzzers with different seed corpora to see their advantages and disadvantages. However, in practice, researchers and hackers have access to valid seeds so it would be good if at least a few valid seeds are provided for each benchmark. And we could have the evaluation results on empty seed, invalid seeds, valid seeds, and their combination.

BTW, I think it would be good if one more column could be inserted into this benchmark table https://google.github.io/fuzzbench/reference/benchmarks/ showing the input format(s) expected by the benchmark's fuzz driver.

Regards,

Thuan

[jonathanmetzman]

Nice find!
I'd accept a PR fixing these seeds if you'd like to submit one, or I can do it myself, let me know.
I think I'd accept a "format" column as well but I suspect for some targets (libpcap_fuzz_both for example) there isn't a clean answer like "jpg" or "png".

[thuanpv]

Thanks, @jonathanmetzman. Let me work on the PR then.

I think I'd accept a "format" column as well but I suspect for some targets (libpcap_fuzz_both for example) there isn't a clean answer like "jpg" or "png".

Yes, some targets support several input formats (e.g., bloaty works on ELF, Mach-O, and WebAssembly (see https://github.com/google/bloaty)) and I will list all the known ones.

[inferno-chromium]

ping @thuanpv, do you intend to fix seeds here ?