Help: Name resolution from Gluetun or stack sharing container to other containers on network does not work #281
Comments
networkprogrammer
changed the title
|
Hello there! I'll dig more/test it myself tomorrow, but does it work when |
|
@qdm12 , yes, I have tried it with DOT=off and setting DNS_PLAINTEXT_ADDRESS=<local_lan_IP> to no avail. Thanks. |
|
So I did some more digging. On the alpine container that is not sharing the stack with Gluetun, I checked its /etc/resolv.conf config. It points to docker's embedded dns server 127.0.0.11 I then ran the same test on the container that shares the network stack with gluetun and the name resolutions worked. So it seems the DNS server change is causing this change in behavior I am not that familiar with Go or the way the code for gluetun works, to help with code changes. Can you do any config in unbound to send non-FQDN queries towards the built-in dns servers and everything else to DOT ? Thanks. |
|
Hello @networkprogrammer sorry I went short on time; Anyway, thanks for digging. The problem I think of is that if you use the Docker network DNS resolver, it will use it for resolving anything instead of Unbound (i.e. Let me know if you find anything, I'll be happy to add it to the Go code so you can use it through an env variable. |
|
Hi @qdm12 , services:
gluetun:
image: myvpn
container_name: gluetun
cap_add:
- NET_ADMIN
networks:
frontend:
ipv4_address: 172.18.0.100
dns: 172.18.0.100
ports:
- 8000:8000/tcp # Built-in HTTP control server
- 8080:8080/tcp #Qbittorrent
- 53:53/udp
- 53:53/tcpDisabling systemd-resolved on ubuntu. This is needed as port 53 is used by the systemd resolver. I cannot map it to gluetun without disabling the resolver. So However I am failing to get unbound respond to queries from anything outside localhost. With the little Go knowledge and without digging too deep into Gluetun I keep failing when building the image manually. Unbound does not listen to outside queries by default, adding the access-control will allow it to respond to queries from outside. Any guidance on how to get past the test step in the image build? I also initially added multiple Thanks again. |
|
If you feel like fiddling a bit with Go and gluetun:
I'm afk right now, but I'll add you as maintainer so you can easily make a branch/PR and I can help fixing it up. |
|
I'm still trying to find a zero-config change solution though. For now the following should work right?.,..
Although that requires to add a dns entry to your docker configuration. I can always add an env variable to enable this different behavior, but it's not ideal. Maybe an alternative would be to tell Unbound to use the Docker network DNS only for private addresses, but I'm not sure that's possible. If it is, the Go program could detect the original DNS address (before overriding it) and set it in the Unbound configuration. That may solve #188 as well. I'll dig into the Unbound configuration options. |
|
Hey @qdm12, Then I build my docker image After that used the image in my dockerfile. Skipping all the irrelevant parts from the service definition. version: "3.7"
services:
gluetun:
image: myvpn
container_name: gluetun
cap_add:
- NET_ADMIN
networks:
frontend:
ipv4_address: 172.18.0.100
dns: 172.18.0.100
environment:
- DNS_KEEP_NAMESERVER=onSo what that gives me is the ability to not only query the local services, but also use So the big thing here is to allow query from the subnet tied to the main/default interface. In my case, I statically assigned the network in the I also noticed that depending on where I place the access-control directive, the build-test failed. Thanks, |
|
No problem, thanks a ton for stretching this out in all the directions! I can definitely test it myself too, so it should be easy to integrate nicely. Allow me 1 to 2 days so I can get to it, I'm a bit over-busy currently unfortunately, but I can't wait to fix this up! Plus this should be how it is behaves natively imo. |
|
Thank you for looking into this. |
|
I'm still testing things out, I would ideally like it to work without having to specify the DNS at the Docker configuration level. Plus, since Unbound blocks i.e. malicious hostnames, I cannot just add the local DNS below Unbound as this would resolve blocked hostnames. Maybe I'm asking for too much 😅 I'll let you know what I find. |
|
So the way I am thinking of solving this for myself is to just allow unbound to listen on the default interface and localhost. This is the key to get this working. Ideally this would be done programmatically during run-time. The rest of the config is already provided by Gluetun's env variables or docker-compose directives. networks:
frontend:
ipv4_address: 172.18.0.100
dns: 172.18.0.100
environment:
- DNS_KEEP_NAMESERVER=onFor those who are OK with the way things are, nothing needs to change. To add this as a feature, we can provide users with some sort of env variable based switch. This would be a code change. That is all we need to get local and internet names resolve. The additional feature enabled with this config would be that other containers and hosts on the network (not just the docker network) can now use gluetun as a DOT resolution host. So Gluetun also becomes a local DNS server that provides DOT over VPN :) Let me know if this helps. |
|
I have a (convoluted) solution in mind which relies 'less' on the OS:
I'm still playing around with @networkprogrammer Thanks for the suggestions! Let me change that interface Unbound is listening on to the default interface, having a DNS over TLS server through the tunnel is definitely interesting 😄 |
Nice work! |
|
So I tested and everything seems to work as expected. To get this to work, I have to set the I'm ok with closing this issue. Thank you again, for the resolution and also getting this awesome project going! |
|
@qdm12 , btw where is the code you did for the DNS server? I am interested is learning go so wanted to see what the code looks like. |
Let me finish (and start haha) that DNS proxy to solve the issue properly. It's good we have workarounds for now, but I would definitely like to fix it properly.
Nowhere yet! I'll get to it in the coming days, I'll tag you and comment here once I have a start of a branch going if you want to pull request review/ask questions 😉 Although that will likely just be a UDP proxy inspecting DNS queries and routing them accordingly (I did a UDP proxy but never fiddled with DNS either). |
|
This is blocked by #289 I think. Do you guys manage to reach other containers from Gluetun in the same Docker network using their IP addresses? |
|
I did a quick test. My setup involves Gluetun and qBittorrent(qbt) sharing the network stack. All other containers are on the same network, but do not share network stacks. From sonarr/radarr etc I can connect to qbt as expected. So I got on the Gluetun container and as a quick test, flushed iptables and then set the default policy to |
|
So Gluetun/qbt -> other containers is not working. Iptables is blocking |
|
OK, so the problem is with Now Gluetun/qbt can talk to other containers on the network. So we need to allow traffic to the local network. |
|
Nice thanks! That did the trick. I'll get to the DNS proxy this weekend, will let you know. |
|
I asked on Reddit's HomeNetworking subreddit here to see if there is a way to do this natively. Let's wait if there is a solution coming up in the next few hours / days before adding (yet another) server to Gluetun haha (we have 5 so far: http proxy, shadowsocks, control server, unbound and healthcheck server). |
|
So I pulled the latest image and see that DNS has stopped working. Something must have changed. Top used to show unbound. so looks like unbound stopped running. Did I pull a docker image that was in development?
|
|
I get this when I try running unbound manually. |
|
Hello @Maxattax97, The workaround I have detailed there works: Not necessarily ideal, because you have to use FQDN names of containers you want to resolve from gluetun attached containers, ie. instead of "whatever" you need to use "whatever.docker-network-name" but it has been working for me for months flawlessly. Hope that helps... |
You're the best. Thank you for this, it worked great. For me, I didn't need a FQDN to access my container once I added the named network; not sure why it was different. |
Thanks a lot for your message, I was quite stuck by this so I struggled a bit to find the solution! :) Did you put the exact same config as me in the include.conf? By forcing using the domain in the FQDN, I was also making sure no DNS leaks could occur and only the "host.docker-network" FQDN were escaping from the VPN name resolution. Maybe there is a more elegant way to do this like you may have done? One that would only resolve hostname with no FQDN through the docker DNS... |
FYI, I have just done the test and in my environment hostnames alone do not work: |
|
For anybody still struggling... here is my fully working config for this case. Using this, I can access services such as qbittorrent outside the gluetun network via name, and I can also access containers outside of gluetun from within containers such as qbittorrent via name (I hope that makes sense). Essentially doing it like this has allowed me to configure everything just as I would if I wasn't using gluetun. The aliases allow external access to qbittorrent etc from the same network, and I'm not 100% certain but I beleive it was the EDIT: As @mpsarakis mentions below, using |
|
By using DNS_KEEP_NAMESERVER=on, doesn't that mean you keep the DNS configuration untouched and therefore all the name resolution goes through your docker DNS (and not only for container names) so you are in effect leaking all your DNS queries to your internet connection DNS instead of those being protected by your VPN connection. If privacy is not an issue then it should not matter. This assumes I have understood the effect of this actual parameter... |
Potentially, I'm honestly not 100% sure, but yes that could be a concern for some. In my case, this is a non-issue, because all my DNS goes through my own DNS servers which forward all DNS queries on my network through DNS over HTTPS Also, if you just need to access containers INSIDE gluetun using dns names, you can of course omit this variable without any issues |
|
Or you can also do as I have done and simply tweak gluetun unbound DNS server from outside the container (using a bind file) and you will have both functionalities: you will be able to resolve your container names AND all the rest will go through the VPN DNS as it should be. |
I actually missed this, nice solution! I might give that a try later, although I tend to avoid bind mounts in favour of docker volumes for everything, but I'm sure I could make it work with a persistant volume |
@mpsarakis I think there's a small typo, should the forward-addr be 127.0.0.1? |
|
no that's the docker internal DNS 127.0.0.11 |
Ah, I wasn't aware of that, thank you for clarifying! |
I tried this and I still can't access containers by name from gluetun or from containers connected to it... Here is my docker compose: networks:
proxy:
external: true
backend:
internal: true
services:
gluetun:
image: qmcgaw/gluetun:latest
container_name: gluetun
hostname: gluetun
cap_add:
- NET_ADMIN
- SYS_MODULE # to load kernel-level wireguard
devices:
- /dev/net/tun:/dev/net/tun
networks:
- proxy
- backend
environment:
- TZ=xxxxxxx/xxxxxxx
- PUID=1001
- PGID=1001
- WIREGUARD_IMPLEMENTATION=kernelspace
- VPN_SERVICE_PROVIDER=xxxxxxxxxxxxxxxx
- VPN_TYPE=wireguard
- VPN_PORT_FORWARDING=on
- VPN_PORT_FORWARDING_PROVIDER=xxxxxxxxxxxxxxxx
- SERVER_COUNTRIES=xxxxxxxxxxxxxxxx
- PORT_FORWARD_ONLY=on
- WIREGUARD_PRIVATE_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- SHADOWSOCKS=off
- HTTPPROXY=off
- FIREWALL_OUTBOUND_SUBNETS=172.18.0.0/16
volumes:
- /etc/localtime:/etc/localtime:ro
- /lib/modules:/lib/modules:ro # to load kernel level wireguard
- xxxxxx/gluetun:/gluetun
- xxxxxx/ip_port:/tmp/gluetun
- xxxxxx/include.conf:/etc/unbound/include.conf:ro
labels:
- traefik.enable=true
- traefik.docker.network=proxy
- traefik.http.routers.qbittorrent.tls=true
- traefik.http.routers.qbittorrent.rule=Host(`qbittorrent.xxxxxx.xx`)
- traefik.http.routers.qbittorrent.entrypoints=websecure
- traefik.http.routers.qbittorrent.tls.certResolver=letsencrypt
- traefik.http.routers.qbittorrent.tls.options=modern@file
- traefik.http.routers.qbittorrent.middlewares=internal@file
- traefik.http.routers.qbittorrent.service=qbittorrent
- traefik.http.services.qbittorrent.loadbalancer.server.port=8080
restart: always
security_opt:
- no-new-privileges:true
qbittorrent:
image: ghcr.io/hotio/qbittorrent:latest
container_name: qbittorrent
network_mode: "service:gluetun"
environment:
- PUID=1001
- PGID=1001
- UMASK=002
- TZ=xxxxxxx/xxxxxxx
volumes:
- xxxxxx/qbittorrent:/config
- xxxxxx/qbittorrent/scripts:/scripts
- yyyyyyyyyyyy/downloads:/downloads
depends_on:
gluetun:
condition: service_healthy
restart: unless-stopped
security_opt:
- no-new-privileges:true
port-updater:
build: https://github.com/tcj-one/qbittorrent-port-forward-file.git
container_name: port-updater
user: 1001:1001
networks:
- backend
environment:
- QBT_ADDR=http://gluetun:8080
- QBT_USERNAME=xxxxxx
- QBT_PASSWORD=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- PORT_FILE=/config/forwarded_port
volumes:
- xxxxxx/gluetun/ip_port:/config:ro
depends_on:
- qbittorrent
- gluetun
restart: unless-stopped
security_opt:
- no-new-privileges:truethe I would like to access an endpoint on a Betanin container, defined in another docker compose file, and that is also attached to the Then:
/ # ping 172.18.0.20
PING 172.18.0.20 (172.18.0.20): 56 data bytes
64 bytes from 172.18.0.20: seq=0 ttl=64 time=0.268 ms(172.18.0.20 is the IP for the container I want to access) / # ping betanin
ping: bad address 'betanin'and / # ping betanin.proxy
ping: bad address 'betanin.proxy'... What am I missing? :( |
|
I I tried following the solution from @mpsarakis , but I also still can't access containers by name from containers inside gluetun. My include.conf is Connecting to gluetng with Any help would be appreciated :) |
|
@yorah You can see all existing network names with Hope that helps... |
|
Thank you for following up! I checked, and the name of my network is indeed |
|
Additional info, just in case it gives you more ideas:
Am I doing something wrong? When I do ping, does it not use 127.0.0.11, as set in the include.conf file? |
|
you're welcome, yes I use a custom one (bridge type), but I am not sure that's the reason. Maybe I have something to say but my explanations are maybe not using the right technical words :) Is your "cross" container inside the gluetun "stack" ie. using "network_mode: service:gluetun" ? Unbound trick is for gluetun stack containers to contact a container outside the actual gluetun "stack". If "cross" is outside, a ping cross.network_name should work from gluetun, if not there is indeed a problem... |
|
My cross container is NOT inside the gluetun stack. I want to contact it from inside gluetun. I am wondering: how is the include.conf file included in unbound? I just checked the /etc/unbound/unbound.conf file, and there is no |
|
Allright, I found out what the problem was... I was using the image I tried with the @qdm12 sorry if this is wrong to ping you, but it seems the latest is broken in that aspect. Thanks @mpsarakis for helping out! |
|
Sorry, I see, I am also using qmcgaw/gluetun:v3 so that must be why I also don't have the problem. My unbound.conf is indeed normal and including the include.conf file. I don't exactly understand what is the actual version for the "latest" tag, I "think" at some point I found it was pointing to something older than "v3" and that's why I have switched to this tag instead a while ago... |
|
Latest image pulled with the v3 tag does not use unbound anymore. This seems to be linked to #1742, but it breaks the workaround proposed in the comments of this PR. |
|
Latest version to support unbound is v3.39. Workaround with latest version seem to involve having a static IP address for the container you want to access that it outside of the gluetun network (https://github.com/qdm12/gluetun-wiki/blob/main/setup/inter-containers-networking.md). @qdm12 is my understanding correct? |
|
@yorah, that's a pity, hopefully something will be integrated in the image to allow gluetun stack external communication without fixed IPs... |
|
As far as I understand, gluetun v3.40 now integrates qdm12/[email protected] and this application allows these settings to be used: So based on this and the previous workaround logic, we could have something like the following defined in gluetun using env variables: I don't exactly know if there is a mandatory relationship between MIDDLEWARE_SUBSTITUTER_SUBSTITUTIONS and the other two variables, ie. if we can use MIDDLEWARE_SUBSTITUTER_SUBSTITUTIONS alone by itself or not... I have tested both ways and they don't work... Maybe @qdm12 could kindly give us some additional info? |
|
I did a bit of research and using qdm12/[email protected] in standalone (ie. in its container) works perfectly to resolve container names, all MIDDLEWARE_* env variables seems to be handled correctly by the container. However, what I have noticed in the latest gluetun v3.40 container which is supposed to contain qdm12/[email protected] components:
So it seems that some features of qdm12/[email protected] are missing and/or are partially implemented in gluetun v3.40 release for reasons I am unfortunately not able to understand, if they were there I think it would fix the issue we have... @qdm12 is apparently unavailable/slowed down for several understandable reasons so for the time being I will stick to release v3.39 which still uses unbound and therefore allows my fix to work. We'll see in the future what happens... |
|
Came here looking for answers after been struggling with connecting Overseerr (using gluetun) to Plex (not being on gluetun) without success. After some trial and error I finally succeeded with these settings: Before this change I also followed this guide and put Plex on the same network as Gluetun: https://github.com/qdm12/gluetun-wiki/blob/main/setup/inter-containers-networking.md I run Unbound on my firewall with DoT already, so this is perfect for me. I hope it helps someone! |
|
If anyone's interested in running a custom version of this with the feature to set LOCALDNS_RESOLVERS you can take a look at my fork specifically this branch gdsoumya#1 build a custom image with the Dockerfile and use it with the following env. Env to use : DOT_PRIVATE_ADDRESS needs to be overwritten to exclude the docker compose resolved ip subnet range or else the filter middleware rejects the dns queries. In my case I had to remove the 172.16.0.0./12 ip. It has a few more custom changes like a barebones UI to interact with control server(still wip) and changes to the control server api routes(appends /api) and some other changes like on demand VPN start/stop feature which you may ignore if not required. |
I just tried this and can confirm it works for me! Thx a lot! |
and others
TLDR: Unable to resolve containers on the same user-defined network using built-in docker dns.
Is this urgent?
What VPN service provider are you using?
What's the version of the program?
You are running on the bleeding edge of latest!What are you using to run the container?
Extra information
Logs:
Working example from container: alpine
Example from container: gluetun where dns fails
Configuration file:
Host OS: Ubuntu 20.04 LTS
Hello,
I am trying to setup my containers such that I can call them with names. My setup consists of Gluetun on the "frontend" network. Qbittorrent shares the network stack with Gluetun. 2 additional containers exist, Jackett and alpine. As you can see from the logs, from the alpine (test) container, I am able to resolve the names of jackett and gluetun containers.
I am however unable to do this the other way, i.e. resolve the names of jackett or alpine from the gluetun container. I am sure this has something to do with the DOT setup, but I have tried various things to no avail.
192.168.1.0/24 is my local lan. I left it in there so that traffic an talk to local LAN services.
Any assistance would be appreciated.
The text was updated successfully, but these errors were encountered: