The open benchmark platform for GenLayer validators.
Bradbury Gym measures what GenLayer validators can actually do. Each benchmark deploys real intelligent contracts to the network, exercises validator capabilities through consensus, and records the results — no mocks, no simulations, just on-chain truth.
The first benchmark tests vision: can a validator's LLM correctly identify the color of a single-pixel image? It sounds simple. The results are surprising.
Live at gym.genlayer.foundation
GenLayer validators run LLMs to execute intelligent contracts. But not all LLMs are equal — they differ in vision capability, reasoning speed, instruction following, and more. Validators choose which models to run, and the network's overall intelligence depends on those choices.
Bradbury Gym gives the community a way to:
- Measure validator capabilities with reproducible, on-chain benchmarks
- Compare networks (Studionet, Asimov, Bradbury) side by side
- Track how validator intelligence evolves over time
- Identify which validators consistently produce correct results
If you're a validator operator, this tells you how your setup compares. If you're a contract developer, this tells you what you can rely on.
Bradbury Gym GenLayer Network
──────────── ────────────────
1. Create Run POST /api/benchmark
→ Generate N random colors
→ Map each to a unique hash
→ Store in Postgres
2. Per Iteration POST /api/benchmark/execute
→ Deploy VisionBenchmark contract ──→ Contract deployed on-chain
→ Contract fetches image URL ←── GET /api/pixel/{hash}
→ LLM analyzes the image ←── (1x1 solid-color PNG)
→ Validators reach consensus ──→ Result stored on-chain
→ Read contract result ←── get_result()
→ Compare answer vs expected
→ Store result + validator info
3. Dashboard Real-time progress, accuracy stats, per-validator breakdown
Key design: each benchmark iteration deploys a fresh contract. The contract receives a URL to a solid-color pixel image, asks the LLM to identify the color, and stores the answer on-chain. The hash-based URL ensures the contract can't derive the color without actually looking at the image.
The first benchmark included in this repo. Located at contracts/vision_benchmark.py.
class VisionBenchmark(gl.Contract):
image_url: str
color_answer: str
def __init__(self, image_url: str):
# Leader fetches the image and asks the LLM what color it is
# Validators accept the leader's answer (we're testing the leader's LLM)
# Result is stored on-chainWhat it tests: LLM vision — can the model correctly identify a solid color from a 1x1 PNG?
Colors tested: red, green, blue, yellow, purple, orange, white, black
Why it matters: Vision is the foundation of many intelligent contract use cases (verifying images, reading documents, checking visual content). If a validator can't identify a solid color, it can't handle more complex visual tasks.
This is where you come in. Bradbury Gym is designed to grow through community contributions. Every benchmark is a PR.
A benchmark has three parts:
A Python contract in contracts/ that exercises a specific validator capability. It should:
- Accept input that the contract itself cannot predict the answer to
- Use
gl.nondetoperations (web fetches, LLM prompts) that require validator intelligence - Store the result in a readable way via a
get_result()view function
# contracts/your_benchmark.py
from genlayer import *
class YourBenchmark(gl.Contract):
result: str
def __init__(self, test_input: str):
def leader_fn():
# Do something that requires LLM intelligence
answer = gl.nondet.exec_prompt("Your prompt here")
return answer
def validator_fn(leaders_res):
# Decide validation strategy
return isinstance(leaders_res, gl.vm.Return)
self.result = gl.vm.run_nondet_unsafe(leader_fn, validator_fn)
@gl.public.view
def get_result(self) -> dict:
return {"result": self.result}Server-side logic in frontend/src/app/api/ that:
- Creates benchmark runs with test data (inputs + expected outputs)
- Executes iterations by deploying contracts and reading results
- Serves any external data the contract needs (images, text, etc.)
React components in frontend/src/components/ that display results. The existing components (StatsCards, ResultsTable, RunsList) are reusable — you may only need to add benchmark-specific visualization.
Here are capabilities worth measuring. Pick one and build it:
| Benchmark | Tests | Difficulty |
|---|---|---|
| Text Extraction | Can the LLM read text from an image? | Medium |
| Math Reasoning | Can it solve arithmetic/logic problems? | Easy |
| JSON Compliance | Does it return valid JSON when asked? | Easy |
| Instruction Following | Does it respect format constraints precisely? | Medium |
| Web Data Extraction | Can it fetch a page and extract specific data? | Medium |
| Multi-step Reasoning | Can it chain logic across multiple prompts? | Hard |
| Multilingual | Does it handle non-English prompts correctly? | Medium |
| Adversarial Prompts | Can it resist prompt injection in contract inputs? | Hard |
| Image Classification | Can it categorize images beyond solid colors? | Medium |
| Latency Under Load | How fast do validators respond at scale? | Hard |
| Validator Agreement | How often do validators agree with each other? | Medium |
| Semantic Equivalence | Can it judge if two texts mean the same thing? | Hard |
- Fork this repo
- Create your contract in
contracts/ - Add API routes for creating runs and executing iterations
- Add or extend UI components for displaying results
- Test locally against Studionet (
NEXT_PUBLIC_DEFAULT_NETWORK=studionet) - Open a PR with:
- What capability you're testing and why it matters
- Example results from at least one network
- Any new environment variables needed
We'll review, test on Bradbury, and merge.
- Node.js 20+
- A PostgreSQL database (we use Neon)
- A funded GenLayer wallet (for deploying contracts)
cd frontend
npm install
cp .env.example .env.local
# Edit .env.local with your values
npm run dev| Variable | Description | Example |
|---|---|---|
PRIVATE_KEY |
Server-side wallet private key for deploying contracts | 0xabc123... |
DATABASE_URL |
PostgreSQL connection string | postgres://user:pass@host/db |
NEXT_PUBLIC_DEFAULT_NETWORK |
Default network to benchmark | bradbury |
NEXT_PUBLIC_APP_URL |
Public URL (contracts fetch images from here) | http://localhost:3000 |
| Network | Purpose | Explorer |
|---|---|---|
| Studionet | Development & testing | explorer-studio.genlayer.com |
| Asimov | Public testnet | explorer-asimov.genlayer.com |
| Bradbury | Public testnet | explorer-bradbury.genlayer.com |
bradbury-gym/
├── contracts/
│ └── vision_benchmark.py # Vision color detection contract
│
├── frontend/
│ └── src/
│ ├── app/
│ │ ├── api/
│ │ │ ├── benchmark/ # POST: create run, execute iteration
│ │ │ ├── pixel/[hash]/ # GET: serves 1x1 color PNG
│ │ │ └── runs/ # GET: list runs, get run details
│ │ ├── layout.tsx # Root layout, fonts, theme
│ │ └── page.tsx # Dashboard page
│ │
│ ├── components/
│ │ ├── header.tsx # App header
│ │ ├── run-controls.tsx # Network selector, iteration count, run button
│ │ ├── stats-card.tsx # Accuracy/completion stats cards
│ │ ├── results-table.tsx # Per-iteration results with explorer links
│ │ ├── runs-list.tsx # Historical run list
│ │ └── providers.tsx # React Query provider
│ │
│ └── lib/
│ ├── db.ts # PostgreSQL queries & schema
│ ├── colors.ts # Color definitions & utilities
│ ├── png.ts # 1x1 PNG generation (~67 bytes)
│ ├── types.ts # TypeScript type definitions
│ ├── genlayer/client.ts # GenLayer SDK client factories
│ └── hooks/
│ └── use-benchmark.ts # TanStack Query hooks
│
└── README.md
| Layer | Technology |
|---|---|
| Framework | Next.js 16, React 19, TypeScript |
| Styling | Tailwind CSS v4, dark theme |
| Data | TanStack Query, PostgreSQL (Neon) |
| Web3 | genlayer-js SDK |
| Contracts | Python (GenLayer intelligent contracts) |
| Hosting | Vercel |
One contract = one result. Each benchmark iteration deploys a fresh contract. This is intentional — it isolates results, enables per-deployment tracking, and mirrors how real contracts are deployed.
Hash-based data URLs. The benchmark generates random hashes mapped to test data in Postgres. Contracts fetch data via /api/pixel/{hash} — they cannot derive the answer from the URL itself.
Leader-only testing. Validators use a permissive validation function (always agree with the leader). We're measuring the leader's LLM capability, not consensus dynamics. Future benchmarks can test validator agreement separately.
Sequential execution. The frontend sends one iteration at a time rather than in parallel. This avoids Vercel function timeouts and provides real-time progress updates as each result comes in.
Immutable pixel cache. Pixel images are served with 1-year cache headers. Once a contract fetches an image, the color can never change — results are deterministic from that point forward.
MIT
Built for the GenLayer community. If validators are the muscles of the network, this is where they train.