[April 2025]

[Byron]

While the previous month was full of bug reports, this month is full of fixes (fortunately). And there is more….

SHA1 with collision detection

Let's face it, gitoxide had it coming. Using the fastest possible SHA1 implementation was great while not too many people were using it, but with jj using it for Git interactions the requirements changed.

It was quite a lot of contributed work, but now we are finally en-par with Git, which seems to also use a slow SHA1 implementation with collision detection. Unfortunately, this also removes a major performance advantage as it reduces the SHA1 hashing speed by a factor of roughly 3x, a slowdown most noticeable when cloning repositories. In numbers, on my machine I could previously get ~24.5GB/s hashing speed on, and now it's down to 8.5GB/s.

On the bright side, this will definitely increase the motivation of supporting SHA256 sooner, which should hash with up to 2.1GB per core on my machine, nearly twice as fast as SHA1 prior to collision detection.

Finally, let me share the security advisory that motivated the fix, which should help the downstream to upgrade to the latest release.

zlib-rs as the one and only

With zlib-rs we now have a pure-Rust implementation of zlib-ng, which for gitoxide turned out to be 1% faster than zlib-ng, the previously fastest C implementation. That number is an even greater achievement when realizing this is 1% of 3x longer time compared to the SHA1 implementation without collision detection.

Thanks to Josh Triplett we could now deprecate all zlib-related configuration flags as a pure Rust implementation will never clash with C symbol exports. In a future step, the deprecated Cargo features will be removed for a great reduction in configuration complexity.

Improved safe.directory handling

As gitoxide had the luxury of implementing Git from the ground up it could prepare for certain 'concepts' while Git had to tack them on. With that in mind, safe.directory wasn't all that important, to the point where it wasn't used where it should have been.

For one, Git now supports * as wildcard-suffixes, allowing to set a directory prefix as safe, and with this fix gitoxide will do the same. This will also affect Git configuration, which previously wasn't trusted as safe.directory didn't affect it.

Additionally, and unfortunately only with manual testing, I now believe that the fixed implementation is similar to what Git does.

Welcome to the Family

In an effort to unify the maintenance setup of various related projects, as driven by Eliah Kagan, we brought prodash, cargo-smart-release and learning-Rust-with-Gitoxide into the GitoxideLabs organization.
From here they will get more timely security updates and patches, and maybe even some improvements here and there.

Community

An even faster "Blame" with caching

Even though it's not quite done, yet, I thought it was worth highlighting that Fractional has been working on adding caching support for gix blame, with examples showing a 80x speedup!
When merged, gitoxide will be able to support a new generation of investigative tooling that quickly tells users where lines have been changed, and more.

Gix in Cargo

Thanks to the generous support of the Rustweek organisers Christoph Rüßler and me will have the opportunity to join the Critical Infrastructure group of the Unconference. There, I hope to get in touch with users of Gitoxide, and of course, the Cargo team to finally meet in person :). And of course, I'd love to finish the STF application to get a grant supporting gitoxide development, which ideally will be beneficial for zlib-rs as well.
With a grant, feature development could be sped up, and I could work on gitoxide more as well.

Cheers
Sebastian

PS: The latest timesheets can be found here (2025).

[NobodyXu]

In numbers, on my machine I could previously get ~24.5GB/s hashing speed on, and now it's down to 8.5GB/s.

On the bright side, this will definitely increase the motivation of supporting SHA256 sooner, which should hash with up to 2.1GB per core on my machine, nearly twice as fast as SHA1 prior to collision detection.

That number is an even greater achievement when realizing this is 1% of 3x longer time compared to the SHA1 implementation without collision detection

I found this to be confusing, comparing performance across all cpus vs per core.

It would be great if the report has the gitoxide vs git clone before and after sha1 collision detection to see the real world effect, i.e. when cloning crates.io index, the rust repository, or linux repository

[Byron]

I see. The real-world performance for pack verify seems to be reduced by a factor of 2.4x, but this is what I recall without having performed more scientific testing. gitoxide is still much faster when cloning thanks to much better parallelism, but it doesn't give me quite as much joy anymore.

[NobodyXu]

Thanks, it's good to hear that it's still much faster than git!

[alexanderkjall]

Would you mind if I reported the security advisory to https://rustsec.org/ ?

And would it be ok (copyright wise) if I copied your advisory text there? (As github advisories are not under a permissive license)