connect ETIMEDOUT 185.199.111.154:443 when downloading a feature #281
Comments
|
@dsyer Are you behind a proxy at all? Very odd. |
|
No proxy here. It definitely was intermittent. I eventually got it to work, but it still fails sometimes. |
|
Here's a complete log if it helps when I tried to add another feature. This happens about 90% of the time to me when I change the JSON. |
|
Maybe this is the wrong place to ask, but why not use a base image (which is then properly cached) for nix and docker etc. (things that are extracted into "features" in devcontainers)? "Features" take a long time to install, delaying startup of the devcontainer. And then they fail 90% of the time (right now for me anyway). |
|
We maintain quite a few base images already. Features are about enabling arbitrary ecosystems or adding "one more thing." Features end up as Docker layers, so things like cache from work as expected. Some features like Python that require building under certain circumstances will improve with time as well. You can then pick a base, and jump into the ecosystem you want to use. Features then provide needed settings to ensure things like Go work properly in a container-based environment that arbitrary packaging systems can't do. Certainly we could consider more base images, but we're happy for the community to drive and maintain their own as well. Both Dev Container Templates and Features can be community contributed now to highlight the existence of pre-built images. We need to track down why you are experiencing this problem. All of our base images use Features so it's unclear why you're experiencing this while others are not. It might be some sort of issue with GHCR itself. |
|
Quick follow up - where are you located? It could be geo specific. |
UK. 185.199.111.154 is indeed unreachable. But why is it even trying? |
|
That's a GitHub CDN endpoint. It's acquiring the Nix Feature from GHCR. So, there may be a CDN issue in your region and in Codespaces you must end up hitting a slightly different endpoint. I've let folks on the GitHub side know since this sounds like a GH infrastructure hiccup of some sort. |
|
I don't know if it helps but I seem to have more chance of success if I |
|
Yeah, Features are OCI artifacts, so they're cached (along with any layers that were built from them). Doing a prune forces them to be re-acquired which would trip you randomly hitting this CDN IP is my bet. This is all super helpful info! |
|
Hey, @dsyer - Can you run the debug steps at http://github-debug.com/ and email me the results ( I'm working with the GitHub networking team to understand this better. |
|
Sure. I'm travelling this week, so I'll come back to this on Friday (or maybe Monday) when I'm back. |
|
@dsyer Thanks - it does look like something may be up with githubusercontent.com in you region so thank you for bringing this to our attention. |
|
Same error in France with 140.82.121.34:443 |
|
@julien-blanchon Can you follow the directions above? #281 (comment) The networking team needs data to debug. |
|
@joshspicer same error but with 185.199.110.154:443 everytime different packages |
|
The same with 185.199.109.154:443. |
|
The same with 198.51.44.1:443 |
|
The same with 140.82.121.33:443 |
|
Same issue as well with 140.82.121.33:443 |
1 similar comment
|
Same issue as well with 140.82.121.33:443 |
|
Today its me with 140.82.121.34:443 |
|
Oh, I found this club 😄 |
|
Same issue here today with 2606:50c0:8003::154:443 |
I just restart my router, and it works again. I'm not sure what happened |
|
I did nothing and it worked again. I think the "timeout" may be a real "timeout": the server must be overloaded. |
|
Yeah it might be couple of their servers being overloaded and we ended up with a slower server. I tried the same exact thing on a codespace and no issues there(maybe due to the caching they implemented?). Local had the issue for a while and resolved itself this morning but now same issue. |
|
I've solved it by connecting through 4g on my phone and downloading again. |
|
I'm experiencing this on a remote-ssh host behind a proxy. Any idea ? Any additional config required to support proxy on remote-ssh (I already have the http(s)_proxy env vars setupped) ? Everything else seems to work fine. |
|
@joshspicer , I tested a bit more and I can confirm that building a devcontainer behind a proxy without using "features" works fine. As soon as I had a feature (eg.: git), then I get the ETIMEDOUT error for that feature. Should I open a new issue ? |
|
I have the same problem here. trying to setup a new devcontainer (tried ubuntu and debian as a base, same error) Output from the terminal: [45042 ms] Start: Run: docker version --format {{json .}} What I'm wondering is why does it try to pull alpine? I 've selected debian as a base.. I'm on a Win10Pro machine (proxy mode is 'autodetect') , proxy is manually set by me in vscode settings. I also have docker desktop installed and this is working correctly (using powershell commands to build containers). |
|
Adding my solution, I was getting these timeouts even after using cloudfront DNS, I got this issue when moving to a different wifi network with a different ISP. My solution was to switch off from Cloudflares DNS to the ISP default and it worked. My suggestion to others is the same. either flush dns sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponderor switch to cloudflare dns if you prefer https://one.one.one.one/ |
From @dsyer devcontainers/features#228
There appears to potentially be some sort of networking issue going on when using a feature locally. Possibly with GHCR? //cc @jkeech @edgonmsft @samruddhikhandale @joshspicer
Can someone help with some more detailed instructions on how to use this? I tried adding the feature and getting a lot of errors. Different errors, no obvious pattern. This one is common:
From @Chuxel
@dsyer Would you mind raising an issue with your specifics? In particular the base image you are using, OS, architecture, etc. Possible there's a scenario where things are not working as expected. That particular error though sounds like its actually having a hard time downloading the feature itself which could be connectivity related. devcontainers/features#275 is very specific to the universal image, for example.
From @dsyer
That's it really, for the specifics. The JSON is just the default if you add ubuntu and nix. My network connectivity is generally good here locally. Maybe there was a DNS glitch out there somewhere? 185.199.111.154 looks like Github (not gcr.io).
The text was updated successfully, but these errors were encountered: