-
Notifications
You must be signed in to change notification settings - Fork 36
First draft of QAD blog post #388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from 2 commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
9094bce
First draft of QAD blog post
flub f95a480
write a real date?
flub 73d75af
Apply suggestions from code review
flub 50b545f
fix missing word
flub 3fcb920
Change from IP + port to "address"
flub 5a0e2c5
fix footnote refs
flub 548c0be
slightly tone this down
flub 7f7e57a
fix word
flub 7d40dce
Spell this out
flub 872fbef
tone down the scorn on STUN
flub 32cdd6e
remove unjustified stab at DTLS
flub 13faa49
slighly more explicit
flub faf273c
Whole bunch of fixes, rephrasing etc from a full read-through
flub c4b6cd4
change the title
flub 68bee33
spelling
flub 004f3ef
Apply suggestions from code review
flub e51cd97
Apply suggestions from code review
flub 02c6bce
Set publishing date
flub File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,202 @@ | ||
import { BlogPostLayout } from '@/components/BlogPostLayout' | ||
import {ThemeImage} from '@/components/ThemeImage' | ||
|
||
export const post = { | ||
draft: false, | ||
author: 'Floris Bruynooghe', | ||
date: '2025-08-26', | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
title: 'QUIC Address Discovery', | ||
matheus23 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
description: | ||
"Moving STUN into QUIC", | ||
} | ||
|
||
export const metadata = { | ||
title: post.title, | ||
description: post.description, | ||
openGraph: { | ||
title: post.title, | ||
description: post.description, | ||
images: [{ | ||
url: `/api/og?title=Blog&subtitle=${post.title}`, | ||
width: 1200, | ||
height: 630, | ||
alt: post.title, | ||
type: 'image/png', | ||
}], | ||
type: 'article' | ||
} | ||
} | ||
|
||
export default (props) => <BlogPostLayout article={post} {...props} /> | ||
|
||
# Holepunching | ||
|
||
As you probably know, iroh is in the business of holepunching. | ||
And gives you a QUIC connection on top. | ||
The typical scenario is establishing a direct connection between two devices, like laptops or phones, both on different home networks. | ||
Home networks tend to have a [NAT] router in front of them, | ||
and even when using IPv6 tend to block new incoming connections in the same fashion as a NAT router would. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
And to be fair, blocking random incoming connections to a home network is a sensible choice. | ||
|
||
[NAT]: https://en.wikipedia.org/wiki/Network_address_translation | ||
|
||
The simplified theory of how UDP holepunching works is that both endpoints send a packet to each other at the same time. | ||
Both routers see the outgoing datagram first, and when they receive the incoming datagram, it is considered to be same connection and is allowed in. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
To achieve this in practice you need two things: | ||
|
||
- A means of communicating the coordination. | ||
Iroh uses the relay server as a network path between the two endpoints for this. | ||
We explained this in more detail in the [iroh on QUIC Multipath] post. | ||
|
||
[iroh on QUIC Multipath]: https://www.iroh.computer/blog/iroh-on-QUIC-multipath | ||
|
||
- The address the NAT router is going to be using for the other endpoint. | ||
Because this is where you have to send your holepunching datagrams to. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
The second part is often called "address discovery", and it seems an impossible task. | ||
How are we supposed to predict how a random router on the internet is going to behave? | ||
|
||
# NAT Types | ||
|
||
NAT routers have existed for a very long time, | ||
and as the world tried to understand them many words have been wasted classifying and naming them. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
It's a confusing mess. | ||
[RFC 4787] can be used as a jumping point to explore the bewildering number of references to older RFC as well as updates to it. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Practical people today mostly classify NATs in two types however: | ||
|
||
[RFC 4787]: https://datatracker.ietf.org/doc/rfc4787/ | ||
|
||
- Destination Independent | ||
- Destination Endpoint Dependent | ||
|
||
What does this mean? | ||
A NAT router's job is to map an internal IP + port to an external IP + port. | ||
When a new connection is created from inside the network the endpoint decides on the source IP + port. | ||
The NAT router then creates a mapping and sends the datagram from some external IP address and port. | ||
Incoming datagrams to to this external IP + port are then looked up in the mapping table to deliver back to the origial source IP + port of the endpoint. | ||
|
||
For a Destination Endpoint Indenpendent mapping the mapping is very simple: | ||
for each unique source IP + port pair is mapped to one external IP + port pair, | ||
*independently* of the destination IP + port of the datagram. | ||
That means a single source IP + port can send datagrams to many destinations on the internet, | ||
and they will all share the same external IP + port on the NAT router. | ||
This is very convenient for holepunching. | ||
|
||
For a Destination Endpoint Dependent mapping there could be several variations. | ||
However for a home router that typically does only have one external IP address only the external port can change. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
So the NAT router can pick a new port for each destination, even if the source IP + port remains the same. | ||
|
||
Now think back to holepunching: | ||
you need to know the external IP + port the NAT router will map to in order to send the holepunching datagrams to each other at the same time. | ||
With Destination Endpoint *Independent* NAT you can use the information from another connection for this. | ||
Destination Endpoint *Dependent* NAT however makes this much harder. | ||
There are still tricks you can do, but for now iroh does not yet support this. | ||
|
||
|
||
# Reflexive Transport Address | ||
|
||
This brings us to the fancy term "Reflexive Transport Address". | ||
Consider you are a server sitting on the internet and you receive some datagrams from an endpoint behind a NAT router. | ||
The IP header of the received datagram will contain the source IP address, | ||
while the UDP header will contain the source port number. | ||
The IP + port the server will see is the external IP + port of the mapping the NAT router makes. | ||
To send a response you'd send a datagram addressed to this IP + port. | ||
|
||
In oder words, the source IP + port the server *observes*, | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
is the address it sends responses too. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Thus you can build a server that informs a client endpoint about the clients address as observed by the server. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
The the client this is the *Reflexive Transport Address*. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
If the client is behind a NAT router this will be a different address than the client itself is sending from. | ||
So a client can use this to detect if it is behind a NAT. | ||
A client can go even further and use multiple such servers. | ||
Now it can tell if the NAT router is Destination Endpoint Dependent or Destination Endpoint Independent. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
|
||
# Session Traversal Utilities for NAT: STUN | ||
|
||
Naturally such servers have existed for a while. | ||
As part of all the standardisation around audio-video calls in the form of SIP and WebRTC there was a need for endpoints to learn about their reflexive transport addresses. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
For this the STUN spec was created, | ||
initially in [RFC 3489] and several versions later we are now at [RFC 8489] if we didn't miss anything.[^rfc-numbers] | ||
|
||
Not going to lie about it: I've never read the full STUN spec.[^spec-reading] | ||
It contains a lot and can do many things. | ||
And yet, the really useful part is surprisingly small. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
Until version 0.32 iroh used STUN exclusively. | ||
It worked pretty simple: | ||
|
||
- Generate a STUN transaction ID. | ||
- Send a STUN request to a STUN server in a UDP datagram (the iroh relay server). | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
- Wait for a response from the server matching transaction ID. | ||
|
||
That's it. | ||
|
||
So why change working systems? | ||
Let's look at what we don't get from this: | ||
|
||
- Encryption. | ||
While in theory you can encrypt STUN requesets using DTLS it's not something that is done much. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
It's also DTLS... | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
- Reliability. | ||
It's a simple UDP-based protocol. | ||
If the request is lost you eventually time out and need to resend it, very primitive. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
- Congestion Control. | ||
You will be sending application traffic over the same sockets. | ||
STUN happens outside of this however, | ||
which makes packet loss much more likely if the application is busy. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
All of these are things that are solved in QUIC: QUIC is a secure, | ||
reliable transport with advanced congestion control and loss detection. | ||
And we already use it for our application protocol so we won't have two different endpoints sending and receiving on the same socket. | ||
|
||
[RFC 3489]: https://datatracker.ietf.org/doc/html/rfc3489 | ||
[RFC 8489]: https://datatracker.ietf.org/doc/html/rfc8489 | ||
|
||
[rfc-numbers]: In between there was RFC 5389. RFC number cuteness tricks will never stop being cute. | ||
|
||
[spec-reading]: While I *have* read many QUIC RFCs in their entirity, several times. So it's not like I'm adverse to reading lengthy IETF specs. | ||
|
||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
# QUIC Address Discovery | ||
|
||
This is such an obvious idea that someone already wrote it down as an IETF draft (thanks Maarten and Christian!): | ||
https://quicwg.org/address-discovery/draft-ietf-quic-address-discovery.html | ||
|
||
QUIC Address Discovery, or QAD as we call it, is an extension to the QUIC protocol that gets negotiated during the QUIC handshake. | ||
If negotiated the remote side will send you a new OBSERVED_ADDRESS frame containing the reflexive transport address it observed for you. | ||
|
||
One of the cool things is that this can happen regardless of the application protocol being used, | ||
as it happens entirely in QUIC frames. | ||
So you can still use this connection to carry application data. | ||
|
||
Another really nice feature flowing from this is that this isn't a request-response protocol anymore. | ||
QUIC supports connection migration for clients, | ||
e.g. when your NAT router updates the mapping for some reason, | ||
or when you move from a Wifi network to mobile data, | ||
QUIC will detect this and migrate the connection to this new network, | ||
without losing any data or breaking the connection. | ||
And whenever that happens while the QAD extension is negotiated, | ||
a new reflexive transport address is observed and will be sent in a new OBSERVED_ADDRESS frame. | ||
Thus this becomes event-based rather than request response. | ||
flub marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
|
||
# QAD in iroh Relay Servers | ||
|
||
Since iroh 0.32 both iroh and the relay servers have supported, | ||
and used, QAD as well as STUN. | ||
Since the 0.90 release we have switched to QAD exclusively. | ||
|
||
The work is not finished yet though. | ||
iroh still uses a special-purpose QUIC connection for QAD. | ||
At some point we would like to also support making the normal relay connection over QUIC when possible, | ||
in addition to the current HTTPS1.1/WebSocket connection. | ||
This would be one fewer connection to the relay server and truly allow us to benefit from the event-based nature of QAD. | ||
This is something for after the 1.0 release however. | ||
|
||
|
||
** ** | ||
|
||
### Footnotes |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we want to publish this for monday, we should just put 2025-09-01 & then merge. The post will then just self publish on monday.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does still show up in the preview, does the preview work differently?