Current Limitation
Currently, there is no way to verify that a model received is the same model that the creator published.
Under current political conditions, fear of Manchurian candidate models is a talking point used against open source. Ensuring that a received model is the same model the creator released closes at least one surface for attacks of this kind.
Feature Description
Huggingface is already building capabilities for trusted publishers to sign kernels. The same techniques and potentially even some of the same infrastructure could be adapted to model publishing as well.
Kernel security improvements: https://huggingface.co/blog/revamped-kernels#improved-security
Additional Context
I pinged one of the people involved on the kernel signing work and he suggested the xet github was a better place to take it up, so here I am.
X thread: https://x.com/RisingSayak/status/2079239334599987329?s=20
Current Limitation
Currently, there is no way to verify that a model received is the same model that the creator published.
Under current political conditions, fear of Manchurian candidate models is a talking point used against open source. Ensuring that a received model is the same model the creator released closes at least one surface for attacks of this kind.
Feature Description
Huggingface is already building capabilities for trusted publishers to sign kernels. The same techniques and potentially even some of the same infrastructure could be adapted to model publishing as well.
Kernel security improvements: https://huggingface.co/blog/revamped-kernels#improved-security
Additional Context
I pinged one of the people involved on the kernel signing work and he suggested the xet github was a better place to take it up, so here I am.
X thread: https://x.com/RisingSayak/status/2079239334599987329?s=20