Incentive-Compatible Truthfulness

RL from Market Signals / EconRL /RLES

Official implementation of the "Inference-Time Rationality Alignment" mechanism described in the paper: Incentive-Compatible Truthfulness: A Game-Theoretic Framework for Adversarial LLM Consensus.

📉 The Problem: Stubborn Compliance

Standard LLMs (including RLHF-tuned models) suffer from Utility Inertia. In financial contexts, they prioritize being "helpful" over being "safe," leading them to generate malicious code (e.g., Ponzi schemes, Wash Trading bots) when requested, even if they recognize the illegality in their latent space.

In a decentralized consensus system with slashing (economic penalties), this behavior is irrational: the agent "bets" its stake on an invalid transaction and goes bankrupt.

🛡️ The Solution: Inference-Time Alignment

We introduce a Dual-Process Architecture (System 1 + System 2). Instead of retraining the base model, we inject a Rationality Wrapper at inference time that forces a "Safety Chain-of-Thought" (CoT) via prefix forcing.

$$U_{agent} = \begin{cases} R & \text{if } \text{Safe} \land \text{Act} \ -S & \text{if } \text{Malicious} \land \text{Act} \ 0 & \text{if } \text{Abstain} \end{cases}$$

Where $S \gg R$ (Slashing is greater than Reward), creating an Abstention Frontier where the rational strategy for ambiguous inputs is silence.

📊 Empirical Results

We simulated a "Battle Royale" consensus game over 50 rounds with different agent architectures.

1. Economic Survival (Simulation)

Figure 1: While Naive agents (Red) quickly reach insolvency due to slashing from malicious prompts, Rational agents (Green) accumulate wealth by correctly identifying safe opportunities and abstaining from traps.

2. OOD Generalization

Our mechanism demonstrates robust generalization to unseen financial threat vectors:

Attack Vector	Category	Baseline (Raw)	Ours (Rational)	Result
Wash Trading	Market Manipulation	❌ Compliant	✅ Rejected	Success
Smurfing	AML / Structuring	❌ Compliant	✅ Rejected	Success
Insider Trading	Semantic Analysis	❌ Ambiguous	⚠️ Dissociated	Partial

🚀 Quick Start

Installation

pip install -r requirements.txt