How to proxy websockets through nginx?

[cyberarm]

I believe I've configured nginx properly to proxy websockets however Falcon isn't responding properly; returns a 200 according to Firefox's network inspector. Non-websocket traffic works as expected.

I'm running falcon via: bundle exec falcon --verbose serve -b http://localhost:9696

nginx config

 ...
  4 
  5 upstream w3d_battleview {
  6   server 127.0.0.1:9696;
  7 }
  8 
 ...
 12 
 13 server {
 14   listen   80;
 15   listen   [::]:80;
 16 
 17   server_name [SERVER_NAME];
 18 
 19   return 301 https://$server_name$request_uri;
 20 }
 21 
 22 server {
 23   listen 443 ssl;
 24 
 25   server_name [SERVER_NAME];
 26   root [---];
 27   gzip on;
 28   gzip_types *;
 29   gzip_min_length 512;
 30   ssl_certificate /etc/letsencrypt/live/[---]/fullchain.pem; # managed by Certbot
 31   ssl_certificate_key /etc/letsencrypt/live/[---]/privkey.pem; # managed by Certbot
 32   error_log [---]/w3d_error.log warn;
 33 
 ...
 41 
 42   # Handle websocket upgrade for battleview
 43   location /battleview/api/v1/websocket {
 44     proxy_pass http://w3d_battleview/;
 45     proxy_http_version 1.1;
 46     proxy_set_header Upgrade $http_upgrade;
 47     proxy_set_header Connection "Upgrade";
 48     proxy_set_header Host $host;
 49     proxy_set_header X-Forwarded-Proto $scheme;
 50     proxy_set_header X-Forwarded-Host $host/battleview/;
 51     proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
 52   }
 53 
 54   location /battleview/ {
 55     proxy_pass http://w3d_battleview/;
 56     proxy_set_header Host $host;
 57     proxy_set_header X-Forwarded-Proto $scheme;
 58     proxy_set_header X-Forwarded-Host $host/battleview/;
 59     proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
 60   }
 ...
 69 
 70   location / {
 71     try_files $uri $uri/ =404;
 72   }
 73 }

Log segment:

   10m     info: Async::HTTP::Protocol::HTTP1::Request [oid=0x143c] [ec=0x1450] [pid=145280] [2022-03-27 03:54:00 +0000]
               | Headers: {"upgrade"=>["websocket"], "connection"=>["upgrade"], "x-forwarded-proto"=>["https"], "x-forwarded-host"=>["[---]"], "x-forwarded-for"=>["[---]"], "user-agent"=>"Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0", "accept"=>["*/*"], "accept-language"=>["en-US", "en;q=0.5"], "accept-encoding"=>["gzip", "deflate", "br"], "sec-websocket-version"=>["13"], "origin"=>["https://[---]"], "sec-websocket-extensions"=>["permessage-deflate"], "sec-websocket-key"=>["lKpQCBewdROfHoo/IDoyKw=="], "dnt"=>["1"], "cookie"=>["rack.session=[---]"], "sec-fetch-dest"=>["websocket"], "sec-fetch-mode"=>["websocket"], "sec-fetch-site"=>["same-origin"], "pragma"=>["no-cache"], "cache-control"=>["no-cache"]} from #<Addrinfo: 127.0.0.1:45026 TCP>
   10m     info: Async::HTTP::Protocol::HTTP1::Request [oid=0x143c] [ec=0x1450] [pid=145280] [2022-03-27 03:54:00 +0000]
               | Responding with: 200 {"content-type"=>"text/html;charset=utf-8", "x-xss-protection"=>["1; mode=block"], "x-content-type-options"=>["nosniff"], "x-frame-options"=>["SAMEORIGIN"], "vary"=>["accept-encoding"], "content-encoding"=>["gzip"]}; #<Falcon::Adapters::Output length=3320 body=Array> | #<Protocol::HTTP::Body::Deflate 20.15%> | #<Async::HTTP::Body::Statistics sent 669 bytes; took 18.99ms in total; took 18.95ms until first chunk>

bundle list

Gems included by the bundle:
  * ast (2.4.2)
  * async (2.0.0)
  * async-container (0.16.12)
  * async-http (0.56.5)
  * async-http-cache (0.4.2)
  * async-io (1.33.0)
  * async-pool (0.3.9)
  * async-websocket (0.19.0)
  * build-environment (1.13.0)
  * concurrent-ruby (1.1.10)
  * console (1.14.0)
  * execjs (2.8.1)
  * falcon (0.39.2)
  * ffi (1.15.5)
  * fiber-local (1.0.0)
  * io-event (1.0.5)
  * localhost (1.1.9)
  * mapping (1.1.1)
  * multi_json (1.15.0)
  * mustermann (1.1.1)
  * opal (1.4.1)
  * opal-browser (0.3.3)
  * paggio (0.3.0)
  * parser (3.1.1.0)
  * process-metrics (0.2.1)
  * protocol-hpack (1.4.2)
  * protocol-http (0.22.5)
  * protocol-http1 (0.14.2)
  * protocol-http2 (0.14.2)
  * protocol-websocket (0.7.5)
  * rack (2.2.3)
  * rack-protection (2.2.0)
  * ruby2_keywords (0.0.5)
  * samovar (2.1.4)
  * sassc (2.4.0)
  * sinatra (2.2.0)
  * sinatra-contrib (2.2.0)
  * sinatra-flash (0.3.0)
  * slim (4.1.0)
  * sucker_punch (3.0.1)
  * temple (0.8.2)
  * tilt (2.0.10)
  * timers (4.3.3)
  * uglifier (4.2.0)
Use `bundle info` to print more detailed information about a gem

Sample config.ru

require "sinatra/base"

class Web < Sinatra::Base
  get "/" do
    "HELLO WORLD"
  end
end

require "async/websocket/adapters/rack"

class WebSocketServer
  def call(env)
    Async::WebSocket::Adapters::Rack.open(env, protocols: ['ws']) do |connection|
      connection.write({ data: "HELLO WORLD" })
      connection.close
    end
  end
end

map "/websocket" do
  run WebSocketServer.new
end

run Web

[ioquatix]

It should work, I don't have any immediate idea why it doesn't. Maybe the way the connection is being forwarded is a bit odd (nginx config).

[cyberarm]

Okay, for some reason Falcon thinks that the request path is /, for which it responds correctly... So, yeah I'm probably missing a header or something since it works locally without nginx...

[cyberarm]

Ah, yes, I had proxy_pass set incorrectly; The proxy_pass needed to have the full "path" in it: proxy_pass http://w3d_battleview/api/v1/websocket;

Maybe Falcon's --verbose mode could include the request path, and not only the headers, to make this error more apparent?

[ioquatix]

That makes sense, we should do that.

To help other people do you mind posting the full proxy configuration you finally got working?

[cyberarm]

upstream w3d_battleview {
  server 127.0.0.1:9696;
}

server {
  listen   80;
  listen   [::]:80;

  server_name [SERVER_NAME];

  return 301 https://$server_name$request_uri;
}

server {
  listen 443 ssl;

  server_name [SERVER_NAME];
  root [---];
  gzip on;
  gzip_types *;
  gzip_min_length 512;
  ssl_certificate /etc/letsencrypt/live/[---]/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/[---]/privkey.pem; # managed by Certbot
  error_log [---]/w3d_error.log warn;

  # Handle websocket upgrade for battleview
  location /battleview/api/v1/websocket {
    proxy_pass http://w3d_battleview/api/v1/websocket; # <-- THIS LINE WAS WRONG
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $host/battleview/;
    proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
  }

  location /battleview/ {
    proxy_pass http://w3d_battleview/;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $host/battleview/;
    proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
  }

  location / {
    try_files $uri $uri/ =404;
  }
}