Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: API benchmark

on:
pull_request:
paths:
- "apps/api/**"
- "benchmarks/**"
- "package.json"
- "package-lock.json"
- ".github/workflows/benchmark.yml"
workflow_dispatch:

jobs:
smoke:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm

- name: Install dependencies
run: npm ci

- name: Run smoke benchmark gate
run: npm run benchmark:smoke
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ node_modules
dist
.env
.env.*
!.env.benchmark.example
coverage
*.log
21 changes: 21 additions & 0 deletions POEM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# The Quiet Ledger

In the branch where small ideas compile,
a marketplace wakes line by line;
profiles bloom in careful style,
and trust becomes a working sign.

A client posts a need in light,
a builder answers with a plan;
between the bid, the test, the write,
the product learns to hold the hand.

The API keeps its doors aware,
the token knows which keys belong;
each guarded route is part of care,
each passing check a steadier song.

So let the board remember this:
good software is not speed alone;
it is the craft behind the risk,
and every fix that brings us home.
2 changes: 1 addition & 1 deletion apps/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"scripts": {
"dev": "node src/server.js",
"start": "node src/server.js",
"test": "node --test src/tests"
"test": "node --test src/tests/*.test.js"
},
"dependencies": {
"cors": "^2.8.5",
Expand Down
2 changes: 2 additions & 0 deletions apps/api/src/middleware/rateLimit.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import rateLimit from "express-rate-limit";
import { env } from "../config/env.js";

export const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000,
limit: 200,
skip: () => env.nodeEnv === "benchmark" || process.env.BENCHMARK_BYPASS_RATE_LIMIT === "true",
standardHeaders: "draft-7",
legacyHeaders: false
});
7 changes: 7 additions & 0 deletions benchmarks/.env.benchmark.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
BENCHMARK_TARGET_HOST=
BENCHMARK_AUTH_TOKEN=
BENCHMARK_REQUESTS=25
BENCHMARK_CONCURRENCY=5
BENCHMARK_TIMEOUT_MS=5000
BENCHMARK_OUTPUT_DIR=benchmarks/results
BENCHMARK_BYPASS_RATE_LIMIT=true
47 changes: 47 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# API Benchmarks

This benchmark suite covers all REST endpoints under `/api/` plus `/health` as a service baseline. It captures latency percentiles, sustained and peak requests per second, error rate, and time to first byte for each endpoint.

## Quick start

```bash
npm run benchmark
```

By default, the runner starts the local Express app on an ephemeral port and writes results to `benchmarks/results/`. To target an already running local or staging server, copy `benchmarks/.env.benchmark.example` to `.env.benchmark` and set `BENCHMARK_TARGET_HOST`.

```bash
cp benchmarks/.env.benchmark.example .env.benchmark
npm run benchmark
```

## Configuration

| Variable | Default | Description |
| --- | --- | --- |
| `BENCHMARK_TARGET_HOST` | local ephemeral API server | Server to benchmark, for example `http://127.0.0.1:4000`. |
| `BENCHMARK_AUTH_TOKEN` | generated by local login endpoint | Dedicated benchmark token for protected routes such as `/api/admin/metrics`. |
| `BENCHMARK_REQUESTS` | `25` | Requests per endpoint for full runs. |
| `BENCHMARK_CONCURRENCY` | `5` | Concurrent requests per endpoint for full runs. |
| `BENCHMARK_TIMEOUT_MS` | `5000` | Per-request timeout in milliseconds. |
| `BENCHMARK_OUTPUT_DIR` | `benchmarks/results` | Directory for JSON and Markdown reports. |
| `BENCHMARK_BYPASS_RATE_LIMIT` | `true` for local benchmark mode | Allows local benchmark runs to measure endpoint behavior instead of rate-limit behavior. Do not set this on staging or production. |

Smoke runs use lower traffic and are intended for CI:

```bash
npm run benchmark:smoke
```

Thresholds live in `benchmarks/thresholds.json`. CI fails when any endpoint exceeds the configured p99 latency or error-rate threshold.

## Output

Each run writes:

- `benchmarks/results/<timestamp>.json`
- `benchmarks/results/<timestamp>.md`
- `benchmarks/results/latest.json`
- `benchmarks/results/latest.md`

The Markdown summary is ready to paste into the pull request description.
213 changes: 213 additions & 0 deletions benchmarks/endpoints.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
export const endpoints = [
{
id: "health",
name: "Health check",
method: "GET",
path: "/health",
expectedStatuses: [200]
},
{
id: "auth-register",
name: "Register user",
method: "POST",
path: "/api/auth/register",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
email: `bench-${Date.now()}-${requestIndex}@example.com`,
password: "benchmark-pass",
role: requestIndex % 2 === 0 ? "client" : "freelancer"
})
},
{
id: "auth-login",
name: "Login user",
method: "POST",
path: "/api/auth/login",
expectedStatuses: [200],
body: () => ({
email: "[email protected]",
password: "benchmark-pass"
})
},
{
id: "auth-oauth-callback",
name: "OAuth callback",
method: "GET",
path: "/api/auth/oauth/github/callback",
expectedStatuses: [200]
},
{
id: "auth-refresh",
name: "Refresh token",
method: "POST",
path: "/api/auth/refresh",
expectedStatuses: [200],
body: () => ({ refreshToken: "benchmark-refresh-token" })
},
{
id: "users-list",
name: "List users",
method: "GET",
path: "/api/users",
expectedStatuses: [200]
},
{
id: "users-create",
name: "Create user",
method: "POST",
path: "/api/users",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
email: `api-user-${Date.now()}-${requestIndex}@example.com`,
name: "Benchmark User",
role: "freelancer",
bio: "Freelance API benchmark profile with realistic text payload size.",
skills: ["typescript", "react", "api-performance"]
})
},
{
id: "jobs-list",
name: "List jobs",
method: "GET",
path: "/api/jobs",
expectedStatuses: [200]
},
{
id: "jobs-create",
name: "Create job",
method: "POST",
path: "/api/jobs",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
title: `Benchmark Marketplace API Build ${requestIndex}`,
description: "Build a production-ready marketplace API module with authentication, search, messaging, and performance instrumentation.",
budgetMin: 500,
budgetMax: 2500,
categoryId: "cat_api",
skills: ["node", "express", "postgres", "performance"]
})
},
{
id: "proposals-list",
name: "List proposals",
method: "GET",
path: "/api/proposals",
expectedStatuses: [200]
},
{
id: "proposals-create",
name: "Create proposal",
method: "POST",
path: "/api/proposals",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
jobId: `job_benchmark_${requestIndex}`,
freelancerId: "usr_benchmark_freelancer",
coverLetter: "I can deliver the API feature with measured performance, tests, and production-ready documentation.",
bidAmount: 1400,
estimatedDays: 7
})
},
{
id: "payments-create",
name: "Create payment",
method: "POST",
path: "/api/payments",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
proposalId: `prp_benchmark_${requestIndex}`,
amount: 1400,
currency: "usd",
description: "Benchmark escrow payment intent"
})
},
{
id: "reviews-list",
name: "List reviews",
method: "GET",
path: "/api/reviews",
expectedStatuses: [200]
},
{
id: "reviews-create",
name: "Create review",
method: "POST",
path: "/api/reviews",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
contractId: `ctr_benchmark_${requestIndex}`,
reviewerId: "usr_benchmark_client",
revieweeId: "usr_benchmark_freelancer",
rating: 5,
comment: "Clear communication, reliable delivery, and measurable API performance improvements."
})
},
{
id: "messages-list",
name: "List messages",
method: "GET",
path: "/api/messages",
expectedStatuses: [200]
},
{
id: "messages-create",
name: "Create message",
method: "POST",
path: "/api/messages",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
conversationId: `conv_benchmark_${requestIndex % 5}`,
senderId: "usr_benchmark_client",
recipientId: "usr_benchmark_freelancer",
message: "Can you share the latest milestone and benchmark evidence for the API work?"
})
},
{
id: "notifications-list",
name: "List notifications",
method: "GET",
path: "/api/notifications",
expectedStatuses: [200]
},
{
id: "notifications-create",
name: "Create notification",
method: "POST",
path: "/api/notifications",
expectedStatuses: [201],
body: ({ requestIndex }) => ({
userId: "usr_benchmark_client",
type: "proposal_received",
title: "New proposal received",
message: `Benchmark notification payload ${requestIndex}`
})
},
{
id: "uploads-create",
name: "Upload file",
method: "POST",
path: "/api/uploads",
expectedStatuses: [201],
formData: ({ requestIndex }) => {
const formData = new FormData();
const content = `benchmark file payload ${requestIndex}\n${"x".repeat(2048)}`;
formData.append("file", new Blob([content], { type: "text/plain" }), `benchmark-${requestIndex}.txt`);
return formData;
}
},
{
id: "search",
name: "Search",
method: "GET",
path: "/api/search?q=typescript%20marketplace%20api",
expectedStatuses: [200]
},
{
id: "admin-metrics",
name: "Admin metrics",
method: "GET",
path: "/api/admin/metrics",
expectedStatuses: [200],
auth: true
}
];
5 changes: 5 additions & 0 deletions benchmarks/results/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
*
!.gitignore
!.gitkeep
!latest.json
!latest.md
1 change: 1 addition & 0 deletions benchmarks/results/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

Loading
Loading