Border router sending socket backpressure using tc

[juagargi]

Summary:

• We use WriteBatch to write packets to the UDP/IP socket. It returns the number of packets successfully enqueued, not distinguishing between temporary EAGAIN (back pressure) or permanent ones. This has to change.
• This WriteBatch behaves better with a batch size of 1, as it waits until the EAGAIN is gone. This is useful for testing.
• The use of the tc command in Linux to simulate a slow line has been wrong in our tests.
  For it to work like we want, we need the following:
  • tc needs to share the namespace of the border router: no tc at the host level, but in the container itself.
  • The border router needs to configure the send buffer size to be as small as possible (see below: tc limit)
  • tc needs to have quite some big "limit", so that all packets that are enqueued in the border router send buffer will fit in the qc, and it won't drop any. This is so because packets queued in the qdisc generally remain charged to the originating socket’s transmit memory. Therefore, a small SO_SNDBUF (send buffer) can cause later sends to encounter EAGAIN until earlier packets leave the qdisc.