consistency-ft · environment research

The Belief Gym

Where can an LLM's beliefs be measured — and trained — under uncertainty, on a small budget? A 14-agent research sweep over benchmarks, hidden-information games, and RL stacks: 70 candidates, 16 deep-verified. The answer turned out to be a gap.

The meta-finding: the thing we want does not exist. Every verified candidate lands in exactly one of three buckets — (a) explicit probabilities scored by proper rules but static and single-turn (Paleka tuples, BLInD, ForecastBench); (b) multi-turn hidden-state games with verifiable rewards but beliefs only implicit in actions (TextArena, Battleship, poker); (c) RL training infrastructure with no belief content at all (verifiers, Environments Hub). Nobody has combined them. The recipe that falls out: use (c) as the vehicle, inject (a)'s elicitation and scoring into (b)'s environments — and prefer environments where the harness already knows the true posterior, so "being Bayesian" earns a dense, per-turn, verifiable reward.

Tier 1 — build on these now

Verdict	What	Why it matters here	Fireworks / 1-GPU
USE-NOW	verifiers (MIT, 4.5k★, pushed daily)	The vehicle. Rewards are arbitrary async Python — a Brier/log-score rubric is ~10 lines; also the gateway to the 100+-env Environments Hub. Build on the v1 API (v0 tutorials are stale).	eval plugs in via base_url today; GRPO trainer if we self-host
ADAPT ★	Collaborative Battleship (MIT, ICLR'26)	Sequential evidence + active information gathering; the harness computes a near-exact Monte-Carlo posterior over hidden boards. Our addition: elicit the model's stated P(hit)/P(ship at X) and score it against the harness posterior, plus expected-information-gain rewards for question choice.	OpenAI-SDK client — 2-line switch · yes
ADAPT	BayesBench (MIT code, arXiv:2606.30850)	Bookbag-and-poker-chips for LLMs: multi-turn evidence with closed-form true posteriors (Beta-Bernoulli coins; MovieLens mixture). Per-turn distance-to-posterior is exactly the "within-market updating" skill the futarchy experiments showed missing. Reuse envs + metrics; swap logprob probes for verbalized probabilities.	reimplement elicitation (ships vLLM-only) · yes
ADAPT no license	BLInD (AAAI'25)	Verbalized Bayes nets with pgmpy-exact ground-truth posteriors and a seedable generator — infinite, perfectly verifiable conditional-probability rewards. Single-turn: curriculum stage 1, not the destination. Ask authors for a license grant.	yes · yes

Tier 2 — the forecasting lane (overlaps our Paleka harness)

What	Note
OpenForesight / OpenForecaster (MIT)	Strategic flag: accuracy-RL on their 52.7k synthesized questions already cut Paleka arbitrage violations 43.5%. Our positioning: they need resolved outcomes; our KL-anchor lane is label-free and trains on unresolved questions. Reuse their dataset, offline-news retrieval, and reward shapes; skip their multi-GPU VeRL trainer.
ForecastBench (MIT / CC BY-SA)	Canonical contamination-free leaderboard; resolved-pairs repo = offline reward data; submit our tuned model bi-weekly as the public out-of-distribution eval.
Metaculus AIB + forecasting-tools (MIT)	Five minutes to a live tournament bot (litellm → Fireworks works); proper peer scores; weeks-long resolution makes it a deployment eval, not an RL loop.
Manifold Markets API	Bots and academic model-training explicitly permitted; replay historical dumps for training; pair P&L with Brier — play-money prices are noisy.
Skipped after verification	TruthTensor (no code, crypto framing) · Prophet Arena (platform closed; scoring lib unlicensed) · PolyBench (unlicensed, dormant since March).

Tier 3 — implicit-belief games (thesis evidence, weaker training fit)

SPIRAL (MIT) — self-play RL on Kuhn poker transfers to math reasoning: the strongest published evidence that game-RL trains belief machinery. But no probability elicitation, full-parameter training on ~8×H100, cannot take an API-served policy. WATCH — cite, don't build on.
TextArena (MIT, very active) — 100+ text games including belief-games; a Fireworks agent is literally OpenAIAgent(base_url=…). Terminal win/loss only — needs our elicitation layer on top.
UnstableBaselines (MIT) — LoRA-first self-play trainer for TextArena; only relevant if training on local weights (e.g. a borrowed DGX), since the policy must be locally vLLM-served.
Not deep-verified but noted: AvalonBench / Werewolf-Plus / MindGames (social deduction, heavier), Hanabi (cooperative beliefs, non-LLM tooling), PokerBench (solver-labeled SFT), reasoning-gym (procedural probability tasks, GRPO-ready), Rewarding Doubt (PPO on a log-score betting game — the closest published "wager your credence" recipe), ConfTuner (differentiable tokenized-Brier loss), KellyBench (bankroll sizing), ABIDES / TwinMarket / AucArena / Concordia (econ sims — beliefs load-bearing but tangled with strategy).

The curriculum this implies

Each stage adds one difficulty axis while the reward stays a proper scoring rule against a known or eventual truth. Stage 4 is our own futarchy-fi/simulations venue — the bridge in both directions.

Full text with verdict details: ENVIRONMENTS.md · model dashboard · RLEF × Forecasting proposal · futarchy bridge. Method: 6-angle workflow sweep (forecasting arenas, hidden-info games, Bayes benchmarks, RL stacks, wagering games, econ sims) → dedup → per-candidate repo verification, 2026-08-19.