Skip to content

ECS Image Handler Benchmark Report

Jump to bottom
wchaws edited this page Sep 7, 2023 · 1 revision

关于 https://github.com/wchaws/serverless-image-handler/pull/166 的压测记录

工具

https://github.com/tsenart/vegeta

测试环境

  1. AWS ECS fargate 4 vCPU 8 GB memory
  2. 固定 10 containers
  3. Region us-west-2
  4. 环境变量 
    {
  "env": {
    "SHARP_QUEUE_LIMIT": "1",
    "CACHE_TTL_SEC": "300",  // The cache ttl in seconds, Note that stale items are NOT preemptively removed by default. https://github.com/isaacs/node-lru-cache#ttl
    "CACHE_MAX_ITEMS": "10000", // The max cached keys count.
    "CACHE_MAX_SIZE_MB": "1024" // The max cached size in mb.
  }
}

测试方法

  1. 切换到指定 git tag 版本。
  2. 每次测试前先确保容器都是全新创建的（通过 ECS scale down 再 scale up）。
  3. 使用 ALB Endpoint 作为测试地址（避免 CloudFront 缓存，确保流量回源至 ECS 机群）。
  4. 生成 100 个测试数据。
  5. 连续测试两次：第一次 RPS 1000，第二次 RPS 2000，每次测 5min。
  6. 测试发起方为 us-west-2 的一台 EC2。
  7. 记录测试结果。

测试总结

RPS 1000 v6.7.0-api-adapter-20230703（升级后）：

  1. P99 latency 从 244.641ms 降低至 9.714ms，降低了约 96%。
  2. Pmax latency 从 3.535s 降低至 403.268ms，降低了约 89%。
  3. 502 BadGateway 问题消除。
  4. Success Rate 从 55.29 提升至 99.90%。
  5. 峰值 CPU 从 86% 降低至 14%

另外，当 RPS 从 1000 提升至 2000 后：

  • v6.7.0-api-adapter-20230703：Success rate 没有明显降低，基本维持 99%。
  • v6.6.0-api-adapter-20230227：Success rate 从 55.29% 降低至 26.88%。

测试数据

创建一个 vegeta-urls.txt 文件，其中包含 100 个不同的 resize（宽度从 100 到 199 不等）请求。具体如下：

GET http://<ALB_DOMAIN>/example.jpg?x-oss-process=image/resize,w_100/quality,q_80/auto-orient,0/interlace,1/format,jpg
GET http://<ALB_DOMAIN>/example.jpg?x-oss-process=image/resize,w_101/quality,q_80/auto-orient,0/interlace,1/format,jpg
GET http://<ALB_DOMAIN>/example.jpg?x-oss-process=image/resize,w_102/quality,q_80/auto-orient,0/interlace,1/format,jpg
...
GET http://<ALB_DOMAIN>/example.jpg?x-oss-process=image/resize,w_199/quality,q_80/auto-orient,0/interlace,1/format,jpg

v6.7.0-api-adapter-20230703 测试结果

# ROUND 1: RPS 1000
$ cat vegeta-urls.txt | vegeta attack -duration=5m -rate=1000/s -timeout=300s -format=http | tee result-v6.7.0-api-adapter-20230703.bin | vegeta report
Requests      [total, rate, throughput]         300000, 1000.00, 998.97
Duration      [total, attack, wait]             5m0s, 5m0s, 3.535ms
Latencies     [min, mean, 50, 90, 95, 99, max]  673.006µs, 2.021ms, 1.262ms, 2.163ms, 3.627ms, 9.714ms, 403.268ms
Bytes In      [total, mean]                     974829932, 3249.43
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           99.90%
Status Codes  [code:count]                      200:299695  429:305  
Error Set:
429 Too Many Requests

# ROUND 2: RPS 2000
$ cat vegeta-urls.txt | vegeta attack -duration=5m -rate=2000/s -timeout=300s -format=http | tee result-v6.7.0-api-adapter-20230703.bin | vegeta report -every 10s
Requests      [total, rate, throughput]         600000, 2000.00, 1995.74
Duration      [total, attack, wait]             5m0s, 5m0s, 1.454ms
Latencies     [min, mean, 50, 90, 95, 99, max]  605.763µs, 1.71ms, 1.312ms, 1.823ms, 1.958ms, 3.42ms, 604.825ms
Bytes In      [total, mean]                     1947531733, 3245.89
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           99.79%
Status Codes  [code:count]                      200:598723  429:1277  
Error Set:
429 Too Many Requests

Memory 利用率（RPS 1000）

CPU 利用率（RPS 1000）

v6.6.0-api-adapter-20230227 测试结果

# ROUND 1: RPS 1000
$ cat vegeta-urls.txt | vegeta attack -duration=5m -rate=1000/s -timeout=300s -format=http | tee result-old.bin | vegeta report -every 10s
Requests      [total, rate, throughput]         300000, 1000.00, 547.48
Duration      [total, attack, wait]             5m3s, 5m0s, 2.983s
Latencies     [min, mean, 50, 90, 95, 99, max]  669.755µs, 82.391ms, 97.163ms, 180.192ms, 199.723ms, 244.641ms, 3.535s
Bytes In      [total, mean]                     546836181, 1822.79
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           55.29%
Status Codes  [code:count]                      200:165876  429:134123  502:1  
Error Set:
429 Too Many Requests
502 Bad Gateway

# ROUND 2: RPS 2000
$ cat vegeta-urls.txt | vegeta attack -duration=5m -rate=2000/s -timeout=300s -format=http | tee result-old.bin | vegeta report -every 10s
Requests      [total, rate, throughput]         600000, 2000.00, 536.70
Duration      [total, attack, wait]             5m0s, 5m0s, 461.772ms
Latencies     [min, mean, 50, 90, 95, 99, max]  634.901µs, 64.282ms, 2.669ms, 250.465ms, 295.412ms, 384.858ms, 3.614s
Bytes In      [total, mean]                     549190296, 915.32
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           26.88%
Status Codes  [code:count]                      200:161257  429:438736  502:7  
Error Set:
429 Too Many Requests
502 Bad Gateway

Memory 利用率（RPS 1000 和 RPS 2000）

CPU 利用率（RPS 1000 和 RPS 2000）

Clone this wiki locally