Skip to content

RTMP writer connection not draining recv-q #5776

@hnicke

Description

@hnicke

Which version are you using?

v1.18.2

Which operating system are you using?

Linux amd64 Docker

Describe the issue

Summary

I am seeing reproducible long-running RTMP connection failures when a liquidsoap client reads an RTMP stream (using input.rtmp) from MediaMTX.

The failure pattern suggests that MediaMTX does not continuously drain client-to-server RTMP control messages on an RTMP reader connection. Over time, the server-side Recv-Q grows monotonically. Once it reaches the default sysctl net.ipv4.tcp_rmem recv-q size, the same connection stalls: server-side Send-Q rapidly grows to several MiB (client might be blocked during control message send), then MediaMTX eventually closes the reader with a write tcp ... i/o timeout, and the client reconnects immediately.

This looks like the RTMP writer does not sufficiently read/drain the reverse direction from the client.

Environment

  • MediaMTX version: v1.18.2
  • Client: Liquidsoap v2.4.4 using ffmpeg/libavformat RTMP input
  • Transport: RTMP over localhost
  • Connection: 127.0.0.1:1935 -> 127.0.0.1:<client-port>
  • Runtime: Kubernetes (talos) pod, MediaMTX and Liquidsoap each running in containers in the same pod
  • The issue happens on a long-running live stream after several hours.

Topology

The problematic connection is the local MediaMTX-to-Liquidsoap reader connection:

MediaMTX:   127.0.0.1:1935
Client:     127.0.0.1:<ephemeral-port>

Observed MediaMTX log

When the failure occurs, MediaMTX logs:

[RTMP] [conn [::1]:58382] closed: write tcp [::1]:1935->[::1]:58382: i/o timeout

Immediately after that, a new RTMP reader connection is opened by the client:

[RTMP] [conn [::1]:50920] opened
[RTMP] [conn [::1]:50920] is reading from path '<path>/feed', 2 tracks (H264, MPEG-4 Audio)

The Liquidsoap/ffmpeg side reports:

[input.ffmpeg:2] Feeding failed: Avutil.Error(Input/output error)

Socket queue evidence

The server-side socket for the RTMP connection shows a slowly growing Recv-Q (in my case, ~5b/s).
Then the connection suddenly stalls. Send-Q grows rapidly while Recv-Q stays at roughly the same value.

Using ss we can see the buffer filling up. In this instance, it cant grow more than 119992 bytes.
Every line represents ~0.5s.

State   recv-q   sent-q
ESTAB 119968 13587     [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119968 445       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119976 434       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119976 0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119976 0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119984 13750     [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119984 0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119984 0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 2553      [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 419       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 5402      [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 408687    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 790842    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 1333846    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 1577488    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 2045189    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 2674596    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 2808229    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3198148    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3631788    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 119992 3884456    [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:37540
ESTAB 0      420       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 0      0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 0      0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 0      418       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 0      0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 0      0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 16     0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 16     0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 16     0         [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 16     13983     [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 24     57594     [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724
ESTAB 24     427       [::ffff:127.0.0.1]:1935     [::ffff:127.0.0.1]:51724


## Packet capture evidence

I captured the loopback RTMP traffic with `tcpdump`.

In the client-to-server direction, the payload is very small compared to the media direction. The reverse direction appears to consist of RTMP control/acknowledgement payloads, not media data.

The capture shows:

```text
127.0.0.1:<client-port> -> 127.0.0.1:1935

sending small RTMP payloads back to MediaMTX. These appear to be RTMP acknowledgement/control messages from the ffmpeg/libavformat RTMP client.

The important distinction is:

  • TCP ACK packets without payload do not increase Recv-Q.
  • The growing Recv-Q must therefore come from TCP packets with payload.
  • The payload appears to be RTMP-level client-to-server acknowledgement/control traffic.

This matches the observed behavior: the RTMP client sends small reverse-channel control messages, but MediaMTX appears not to drain them from the socket.

Expected behavior

MediaMTX should continue reading/draining client-to-server RTMP control messages on RTMP reader connections for the lifetime of the connection.

A long-running RTMP reader should not accumulate unread client-to-server payload in the server-side receive queue.

Actual behavior

The server-side Recv-Q for the RTMP reader connection grows monotonically over time.

Once it reaches around ~120 KiB in my test, the RTMP reader stalls:

  1. Recv-Q stops around ~119-120 KiB.
  2. Send-Q rapidly grows to several MiB.
  3. MediaMTX closes the reader with write tcp ... i/o timeout.
  4. The liquidsoap ffmpeg/libavformat client reports an input/output error.
  5. The client reconnects immediately.

Workaround

Increase sysctl net.ipv4.tcp_rmem. (the value in the middle).

Describe how to replicate the issue

  1. download the slate.mp4 video sample as sample.mp4
  2. save below docker compose stack in docker-compose.yml
  3. run docker compose up

You should see output like below. Note the third column (recv-q), which monotonically grows.

mediamtx-1    | 2026/05/16 21:41:27 INF [path live/sink] stream is available and online, 2 tracks (H264, MPEG-4 Audio)
mediamtx-1    | 2026/05/16 21:41:27 INF [RTMP] [conn 127.0.0.1:38822] is publishing to path 'live/sink'
ss-1          | ESTAB 12     12511  [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 12     0      [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 12     0      [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 20     12511  [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 20     0      [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 20     12     [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 28     12511  [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 28     0      [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762
ss-1          | ESTAB 28     9      [::ffff:127.0.0.1]:1935 [::ffff:127.0.0.1]:41762

docker-compose.yml:

services:
  mediamtx:
    image: bluenviron/mediamtx:1.18.2
    command: /mediamtx.yml
    configs:
      - source: mediamtx.yml
        target: /mediamtx.yml
    network_mode: host
    working_dir: /pcap
    volumes:
      - ./pcap:/pcap
  liquidsoap:
    image: savonet/liquidsoap:v2.4.4
    command: liquidsoap /liquidsoap.liq
    configs:
      - source: liquidsoap.liq
        target: /liquidsoap.liq
    depends_on: 
      - mediamtx
    network_mode: host
  ffmpeg:
    image: jrottenberg/ffmpeg:8-scratch
    command: -re -stream_loop -1 -i /slate.mp4 -c:v copy -c:a copy -flvflags no_duration_filesize -f flv  rtmp://localhost:1935/live/feed
    volumes:
      - ./slate.mp4:/slate.mp4
    depends_on:
      - mediamtx
    network_mode: host
  ss:
    image: tazhate/k8s-debug-container-net
    command: |
      bash -c 'ss -t "src 127.0.0.1:1935"; while :; do ss -Ht "src 127.0.0.1:1935" | grep -v "ESTAB 0"; sleep 1; done'
    network_mode: host
    depends_on:
      - liquidsoap
      - ffmpeg
configs:
  mediamtx.yml:
    content: |
      logLevel: debug
      rtmp: yes
      paths:
        live/feed:
        live/sink:
      dumpPackets: true
  liquidsoap.liq:
    content: |
      live = input.rtmp(
        listen=false,
        "rtmp://localhost:1935/live/feed",
      )
      
      output.url(
        fallible=true,
        %ffmpeg(
          format="flv",
          %audio.copy,
          %video.copy
        ),
        url="rtmp://localhost:1935/live/sink",
        live
      )

MediaMTX configuration

logLevel: debug
rtmp: yes
paths:
  live/feed:
  live/sink:
dumpPackets: true

MediaMTX logs

No response

Packet dump

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions