marten RPS about 300, performance is too poor than the http demo on crystal home page(RPS 70k ~ 80k+)

[forchid]

Version info: marten-0.6.1, crystal-1.18.2-msvc, windows x64

The marten test source

The test case

>server --log-level error

Here -n only uses 1000 in the ab test for the time too long in the martin test.

>ab -n 1000 -k -c 50 http://127.0.0.1:8000/
This is ApacheBench, Version 2.3 <$Revision: 1903618 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /
Document Length:        36 bytes

Concurrency Level:      50
Time taken for tests:   3.035 seconds
Complete requests:      1000
Failed requests:        0
Keep-Alive requests:    1000
Total transferred:      538000 bytes
HTML transferred:       36000 bytes
Requests per second:    329.44 [#/sec] (mean)
Time per request:       151.774 [ms] (mean)
Time per request:       3.035 [ms] (mean, across all concurrent requests)
Transfer rate:          173.08 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     5  148 343.0    100    3025
Waiting:        5  148 343.0    100    3025
Total:          5  148 343.0    100    3025

Percentage of the requests served within a certain time (ms)
  50%    100
  66%    112
  75%    120
  80%    126
  90%    133
  95%    138
  98%   1405
  99%   2479
 100%   3025 (longest request)

The test source

require "time"

class HomeHandler < Marten::Handler
  def get
    respond("#{Time.utc} Hello World!")
  end
end

The crystal http demo

The test case

>ab -n 100000 -k -c 50 http://127.0.0.1:8081/
This is ApacheBench, Version 2.3 <$Revision: 1903618 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8081

Document Path:          /
Document Length:        45 bytes

Concurrency Level:      50
Time taken for tests:   1.410 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      13400000 bytes
HTML transferred:       4500000 bytes
Requests per second:    70938.69 [#/sec] (mean)
Time per request:       0.705 [ms] (mean)
Time per request:       0.014 [ms] (mean, across all concurrent requests)
Transfer rate:          9282.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       5
Processing:     0    1   1.7      0      10
Waiting:        0    1   1.7      0      10
Total:          0    1   1.7      0      10

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      5
  95%      5
  98%      5
  99%      5
 100%     10 (longest request)

The test source

# A very basic HTTP server

require "http/server/handler"
require "time"

# RPS 85k+: perfect!
class HelloHandler
    include HTTP::Handler

    def call(context)
      context.response.content_type = "text/plain"
      context.response.print "#{Time.utc} Hello crystal, got #{context.request.path}!"
      #context.response.close
    end
end

server = HTTP::Server.new([HelloHandler.new])

address = server.bind_tcp(8081)
puts "Listening on http://#{address}"

# This call blocks until the process is terminated
server.listen

[forchid]

I find that there is the response header Set-Cookie sessionid in marten test by curl -v. Can this cookie lead to 200 times slower than the http demo on crystal home?

>curl -v  http://127.0.0.1:8000/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Referrer-Policy: same-origin
< X-Frame-Options: DENY
< VARY: Cookie
< Content-Type: text/html
< Content-Length: 36
< Set-Cookie: sessionid=eyJfbWFydGVuIjp7InZhbHVlIjoiTlRGa05qSmtZV1V6TlRsak5tWmtNamczTVdJMFlqaGpaVEkzTjJSa1l6TmxORE00TTJZMU9UUXpNVGd4TldZM05tUTBZV1kwTW1JelpUSmpaV1U0WXc9PSIsImV4cGlyZXMiOiIyMDI1LTEyLTE2VDAyOjM1OjEwWiJ9fQ==--2214429393723a00469faedf4705b5204fcf0e709af3a6353b50331cdf25dbcb; path=/; expires=Tue, 16 Dec 2025 02:35:10 GMT; SameSite=Lax
<
2025-12-02 02:35:10 UTC Hello World!* Connection #0 to host 127.0.0.1 left intact

[ellmetha]

I converted this to a discussion as this is not a bug per se.

I don't really know the conditions under which your test was performed (which settings, which middlewares, which environment, etc - you would need to share your exact project structure for us to try to reproduce), but if you are curious about benchmarks you can see that Marten performs fairly well in this one that we regularly update with new releases (~38k requests/sec).

[forchid]

the conditions
@ellmetha The marten tutorial:

1. marten new project hello;
2. shards install
3. writing a first handler

require "time"

class HomeHandler < Marten::Handler
  def get
    respond("#{Time.utc} Hello World!")
  end
end

4. config.debug = false in config\settings\development.cr
5. crystal build --release src/server.cr
6. run server server --log-level error

The marten project files in a zip file:
hello.zip

[ellmetha]

Ok. Tried it with a fresh project:

$ ab -n 1000 -k -c 50 http://127.0.0.1:8000/
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /
Document Length:        36 bytes

Concurrency Level:      50
Time taken for tests:   0.050 seconds
Complete requests:      1000
Failed requests:        0
Keep-Alive requests:    1000
Total transferred:      538000 bytes
HTML transferred:       36000 bytes
Requests per second:    20091.62 [#/sec] (mean)
Time per request:       2.489 [ms] (mean)
Time per request:       0.050 [ms] (mean, across all concurrent requests)
Transfer rate:          10555.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     0    2   0.9      2       6
Waiting:        0    2   0.9      2       6
Total:          0    2   0.9      2       6

Percentage of the requests served within a certain time (ms)
  50%      2
  66%      2
  75%      3
  80%      3
  90%      4
  95%      5
  98%      5
  99%      6
 100%      6 (longest request)

And using the options you used for the Crystal demo:

$ ab -n 100000 -k -c 50 http://127.0.0.1:8000/
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /
Document Length:        36 bytes

Concurrency Level:      50
Time taken for tests:   2.965 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      53800000 bytes
HTML transferred:       3600000 bytes
Requests per second:    33723.70 [#/sec] (mean)
Time per request:       1.483 [ms] (mean)
Time per request:       0.030 [ms] (mean, across all concurrent requests)
Transfer rate:          17718.11 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       2
Processing:     0    1   0.2      1       6
Waiting:        0    1   0.2      1       6
Total:          0    1   0.2      1       6

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      2
  95%      2
  98%      2
  99%      2
 100%      6 (longest request)