UE5 Quilkin Plugin Bug

[nitecon]

What happened:

The Unreal Engine plugin's QuilkinPacketHandler appears to be incorrectly serializing the TArray<uint8> routing token. When appending the token as a suffix to outgoing packets, it consistently skips the first byte of the token.

This causes the Quilkin proxy, when configured with a suffix capture filter, to read a misaligned token. This misaligned token then fails to match the correct token in the proxy's routing table (which was set by our matchmaker), causing all client connection attempts to fail with a "no endpoint matched token" error.

What you expected to happen:

We expect the QuilkinPacketHandler to append the full 16-byte token to the outgoing packet. The proxy should then read this exact 16-byte token from the suffix, find a match in its routing table, and successfully forward the packet to the dedicated game server.

How to reproduce it (as minimally and precisely as possible):

1. Configure a Matchmaker (like agones-pubsub-allocator) to generate a 16-byte token and register it with the Quilkin proxy's routing table.
2. Configure the Quilkin proxy ConfigMap to capture a 16-byte token from the suffix of the packet. (See Custom filters section below for YAML).
3. Configure an Unreal Engine 5.5 client to use the QuilkinConfigSubsystem and QuilkinPacketHandler.
4. Have the client request a token from the Matchmaker.
5. On the client, receive the 16-byte token, decode it from Base64 into a TArray<uint8>, and set it using UQuilkinConfigSubsystem::SetRoutingToken().
6. Attempt to ClientTravel to the Quilkin proxy address.
7. The connection will fail. The client will log a timeout, and the proxy will log a "no endpoint matched token" error.

Anything else we need to know?:

This is a byte-level serialization bug. The system is fully synchronized on a 16-byte token, but the proxy log proves it is reading a token that is offset by one byte.

---

Byte-Level Proof

---

• Token from Matchmaker (The "Truth"):

  • Base64: bFJUU0tMZTRzS1FZYnFvMA==
  • Decoded: lRTSKLe4sKQYbqo0

• Token the UE Client Initializes With (Correct):

  • Log: LogQuilkin: Display: Initialising PacketHandler: ... Routing Token: bFJUU0tMZTRzS1FZYnFvMA==
  • Hex: [0x6C, 0x52, 0x54, 0x53, ... , 0x6F, 0x30]

• Token the Proxy Receives (Incorrect - 1 Byte Offset):

  • Log: ...error":"filter no endpoint matched token \UlRTS0xlNHNLUVlicW8wAA==`"`
  • Decoded: RTSKLe4sKQYbqo0\x00
  • Hex: [0x52, 0x54, 0x53, 0x4B, ... , 0x30, 0x00]

Comparison:

Array	Byte 1	Byte 2	Byte 3	...	Byte 15	Byte 16
Sent	0x6C (l)	0x52 (R)	0x54 (T)	...	0x6F (o)	0x30 (0)
Read	0x52 (R)	0x54 (T)	0x53 (S)	...	0x30 (0)	0x00 (Junk)

This shows the plugin is failing to write the first byte (0x6C) of the token to the packet. The investigation should focus on the QuilkinPacketHandler's PreSend or LowLevelSend implementation, specifically where the RoutingToken TArray is being copied into the packet buffer.

Environment:

• Quilkin version: 0.9.0 (from proxy log {"message":"Starting Quilkin","version":"0.9.0",...)
• Execution environment (binary, container, etc): Container (running in Kubernetes)
• Operating system: Linux (container base)
• Custom filters? (Yes/No - if so, what do they do?):
  Yes, the standard token capture/router configuration.

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: quilkin-xds-filter-config
  data:
    quilkin.yaml: |
      version: v1alpha1
      filters:
        - name: quilkin.filters.capture.v1alpha1.Capture
          config:
            metadataKey: "quilkin.dev/token"
            suffix:
              size: 16
              remove: true
        - name: quilkin.filters.token_router.v1alpha1.TokenRouter
          config:
            metadataKey: "quilkin.dev/token"
  
    Log(s):
  
    Matchmaker (Allocator) Log:
    JSON
  

{"level":"info", ... "token":"bFJUU0tMZTRzS1FZYnFvMA==", ... "message":"controller: updating GameServer with routing token"}

Unreal Client Log:

LogServer: Display: Starting with token: bFJUU0tMZTRzS1FZYnFvMA==
LogServer: Display: Decoded token to 16 bytes, Token: lRTSKLe4sKQYbqo0
LogQuilkin: Display: Initialising PacketHandler: Packet Handling: Enabled, Routing Token: bFJUU0tMZTRzS1FZYnFvMA==
LogQuilkin: Display: 10.0.10.14:7600 exceeded WaitForRead timeout 6 times

Quilkin Proxy Log:
JSON

{"timestamp":"2025-10-25T18:08:49.789103Z","level":"WARN","fields":{"message":"pipeline report","error":"filter no endpoint matched token UlRTS0xlNHNLUVlicW8wAA==","instances":"6"},"target":"quilkin::components::proxy::packet_router","filename":"src/components/proxy/packet_router.rs","threadId":"ThreadId(2)"}

Others: N/A

[XAMPPRocky]

Thank you for your issue! I would like to verify that it is indeed local to your client, as I believe I had run into this before and remembered it being a load balancer issue.

Can you verify if you run -LogCmds="Quilkin VeryVerbose"? That will give you packet level logging you should see Wrapping packet in Quilkin PDU; version: %d, size: %d, token: %s that is the exact size of the packet and the token at send time.

[nitecon]

The following seemed to effectively solve it for me, and I no longer see the difference in token's and also fixed an issue where dynamically setting a token didn't work as it always used the default token. Happy to submit a PR on this... from v0.9.0 tag:

index 1ddef65c1..0fea9fd96 100644
--- a/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandler.cpp
+++ b/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandler.cpp
@@ -21,7 +21,14 @@
 FQuilkinPacketHandler::FQuilkinPacketHandler()
 {
        auto Settings = UQuilkinConfigSubsystem::Get();
-       RoutingToken = Settings->GetRoutingToken();
+       // Explicitly copy token element-by-element to avoid TArray copy constructor bug
+       TArray<uint8> TempToken = Settings->GetRoutingToken();
+       RoutingToken.Empty();
+       RoutingToken.Reserve(TempToken.Num());
+       for (int32 i = 0; i < TempToken.Num(); ++i)
+       {
+               RoutingToken.Add(TempToken[i]);
+       }
        Enabled = Settings->PacketHandling && Settings->GetEnabled();

        if (UE_LOG_ACTIVE(LogQuilkin, Display))
diff --git a/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandler.h b/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandl
er.h
index d7abc096c..259a3537d 100644
--- a/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandler.h
+++ b/sdks/ue5/Source/Quilkin/Private/QuilkinPacketHandler.h
@@ -20,36 +20,40 @@
 #include "QuilkinLog.h"
 #include "Serialization/BitWriter.h"
 #include "Misc/Base64.h"
+#include "QuilkinSettings.h"

 class FQuilkinPacketHandler
 {
 public:
        FQuilkinPacketHandler();
        bool IsEnabled();
-       bool IsDisabled() {
+
+       bool IsDisabled()
+       {
                return !IsEnabled();
        };

-       template<typename Fn> FORCEINLINE const bool Write(const uint8* Data, int32 CountBytes, int32& BytesSent, Fn WriteToSocket)
+       template <typename Fn> FORCEINLINE const bool Write(const uint8* Data, int32 CountBytes, int32& BytesSent, Fn WriteToSocket)
        {
-               if (IsDisabled()) {
+               if (IsDisabled())
+               {
                        return WriteToSocket(Data, CountBytes, BytesSent);
                }
-               ensureMsgf(RoutingToken.Num() == 16, TEXT("Routing token must be 16 bytes, received %d, proxy connection will fail"), RoutingToken.Num());
-
-               // Add the current packet version.
-               uint8 PacketVersion = 0;
-               int PacketVersionNumBytes = 1;
-               // Reserve enough space for the token and packet version.
-               FBitWriter Packet((CountBytes + RoutingToken.Num() + PacketVersionNumBytes) * 8, true);
+
+               // Get the current routing token from the subsystem (don't use cached value)
+               TArray<uint8> CurrentToken = UQuilkinConfigSubsystem::Get()->GetRoutingToken();
+               ensureMsgf(CurrentToken.Num() == 16, TEXT("Routing token must be 16 bytes, received %d, proxy connection will fail"), CurrentToken.Num());
+
+               // Reserve enough space for the token (no version byte needed for Quilkin 0.9.0+)
+               FBitWriter Packet((CountBytes + CurrentToken.Num()) * 8, true);

                Packet.Serialize((void*)Data, CountBytes);
-               Packet.Serialize(RoutingToken.GetData(), RoutingToken.Num());
-               Packet.Serialize(&PacketVersion, PacketVersionNumBytes);
+               // Serialize the routing token as bytes (not bits) to maintain byte alignment
+               Packet.Serialize(CurrentToken.GetData(), CurrentToken.Num());
                if (UE_LOG_ACTIVE(LogQuilkin, VeryVerbose))
                {
-                       FString Base64Token = FBase64::Encode(RoutingToken);
-                       UE_LOG(LogQuilkin, VeryVerbose, TEXT("Wrapping packet in Quilkin PDU; version: %d, size: %d, token: %s"), PacketVersion, Packet.GetNumBytes(), *Base64Token);
+                       const FString Base64Token = FBase64::Encode(CurrentToken);
+                       UE_LOG(LogQuilkin, VeryVerbose, TEXT("Wrapping packet in Quilkin PDU; size: %d, token: %s"), Packet.GetNumBytes(), *Base64Token);
                }
                return WriteToSocket(Packet.GetData(), Packet.GetNumBytes(), BytesSent);
        }