Cargo does not respect no_proxy environment variable

[lily-mara]

Another issue in the long tradition of HTTP proxies! The fun just never stops. Because cargo does not support private registries, internal packages are stored in our self-hosted internal git host. Since we do use public cargo packages, we still need access to gitlab, which requires a forward proxy. Unfortunately, downloading dependencies from our internal git host requires that there be no forward proxy on the request. This requires the no_proxy environment variable. This way, external requests can be proxied and internal requests can be sent directly. Cargo does not seem to respect this variable and as a result, our internal dependencies can not be installed. This is a big blow to using rust at my company. The no private registry thing was a tough sell, but if this doesn't work I just don't know if it's feasible.

What did I do:

export http_proxy=http://proxy.corp.com
export HTTP_PROXY="${http_proxy}"
export HTTPS_PROXY="${http_proxy}"
export https_proxy="${http_proxy}"
export no_proxy=git.corp.com

cargo build

What happened:

    Updating git repository `https://git.corp.com/tools/logging-helper`
warning: spurious network error (2 tries remaining): failed to receive HTTP 200 response: got 404; class=Net (12)
warning: spurious network error (1 tries remaining): failed to receive HTTP 200 response: got 404; class=Net (12)
error: failed to load source for a dependency on `logging-helper`

Caused by:
  Unable to update https://git.corp.com/tools/logging-helper#b0cb6965

Caused by:
  failed to clone into: /Users/me/.cargo/git/db/logging-helper-68c9501fefa8f883

Caused by:
  failed to receive HTTP 200 response: got 404; class=Net (12)

If I unset all of the proxy variables, then I can download the internal dependencies, but none of the external dependencies.

[alexcrichton]

Thanks for the report! This should be fixed in #6115

[lily-mara]

Thanks for the quick turnaround on this!

[alexcrichton]

Ok we have quite a long discussion in this thread where we've discovered a few things:

• Cargo (transitively via libcurl) already respects NO_PROXY env vars, no changes needed
• The bug is related specifically to .git at the end of the URL vs not. This appears to be GitLab's default behavior

I've reproduced locally using a repo on gitlab. Adding some more debug logging and poking around with gdb has revealed a few things. As more background, Cargo has two downloading backends for git repos: native libgit2 and curl. Cargo relies on libgit2 to get most things right but whenever it detects non-default configuration (like proxies) it'll switch to Cargo's internal curl-based downloader to respect the same settings. (historically libgit2 knew nothing of proxies so this was required to get it to work, but libgit2 has changed a lot in the meantime!)

In any case what I've found is:

• Both backends first make a request to /natemara/cargo-proxy-test/info/refs?service=git-upload-pack.
• Gitlab responds with a 301 redirect to /natemara/cargo-proxy-test.git/info/refs?service=git-upload-pack, notice the .git at the end of cargo-proxy-test
• Both backends resubmit a request to /natemara/cargo-proxy-test/info/refs?service=git-upload-pack
• Now is where behavior starts to diverge
  • Native libgit2 sends the next request to /natemara/cargo-proxy-test.git/git-upload-pack, note the .git at the end. GitLab accepts this
  • Cargo's curl sends the next request to /natemara/cargo-proxy-test/git-upload-pack. GitLab 404's this

When libgit2 processes redirects it'll reset the path we're sending redirects to based on the redirected response. The git2-curl crate that Cargo uses doesn't do this.

So all in all, it looks like the OP of this issue and current title are red herrings. The actual bug here is that git2-curl isn't handling redirects the exact same way as libgit2, which causes GitLab to reject subsequent requests.

The fix is to around here use the effective_url reported by libcurl (URL after all redirects taken into account) to reset self.base_url. That means that the second request sent by Cargo should use the .git suffix as appropriate.

Alternative fixes are:

• Use .git suffixes in Cargo dependencies. This shouldn't be necessary, but would remove one redirect at least!
• Leverage libgit2's native support, but I'm wary to do this because we have more than just proxy settings to copy over to libgit2 (and using curl is much easier as it's all already configured).
• File a bug with GitLab (although I don't really consider this a fix at all, we should fix Cargo)

[lily-mara]

Thank you very much for your patience and understanding. I do apologize for the red-herring issue, and the sidetracking of the pr comment section. Glad we were able to nail the real problem down.

[alexcrichton]

No worries! Was a fun bug to hunt down and actually get to the bottom of, thanks for being so responsive testing out changes!

@ehuss says he's interested in investigating fixing this as well!

[alexcrichton]

Ok git2-curl has been published with 0.8.2 which contains the referenced fix. I've included that update to rust-lang/rust in rust-lang/rust#54919, which once merged means the fix will actually make its way into nightlies