What we learned
Full-label consistency reached its best validation log-loss at epoch 8; accuracy-only peaked at epoch 5. Both became confidently wrong with further training. One-label supervision did not generalize better than trivial baselines, even with consistency.
Exploratory results, one seed and only 20 validation pairs. Checkpoints were selected on this same validation set. The test split remains untouched. These results do not establish robust forecasting gains.
The Boolean reference uses P(single)=½, P(AND)=¼, P(OR)=¾: independent uninformative marginals, without outcome information. Balanced signed forms make its score 0.605939 log-loss and 0.208333 Brier. Its measured consistency residual is zero.
Learning curves
Choose a metric and population. Hover over chart points for exact values; every recorded epoch is in the tables below.
● Accuracy only● Accuracy + consistencyDashed: reference baselines
Full-label supervision
Both arms completed 20 epochs. At epoch 6, saved epoch-5 weights resumed with fresh AdamW state; the vertical marker marks this change. This is not uninterrupted optimization.
One label per group
Accuracy-only was stopped by the user after 7 completed epochs. Consistency completed 20. Each group has one uniformly selected supervised expression, fixed across epochs and identical across arms. Masked-expression scores are from seen training groups, not independent validation.
Stage 1 · engineering checks
Smoke tests, kept separate
Ten singles and ten pairs established that training, cache integrity and checkpoint reload worked. These are different cohorts and a different consistency penalty from the larger pilot; do not combine their scores into its learning curve.
Smoke held-out diagnostics
The pair penalty averaged 27 identities and bounds over all 12 forms. Both pair-trained models still had worse consistency than the initial model on this tiny held-out set. One of the 20 singles traces was reconstructed from saved text and forcibly closed; all 60 pair traces ended naturally.
Earlier variants and integrity results
The earlier frozen-text run reprocessed saved reasoning through trainable weights. It used ten examples and one update, completed in 71 seconds, and is superseded by fixed-KV training. An earlier unspaced YES/NO readout was also superseded; it is not included in the result comparisons. A verbose-prompt attempt hit the reasoning limit on its first example.
Fixed-KV smoke checks found finite gradients in all 310 gradient tensors, unchanged cached tensor hashes, zero checkpoint-reload discrepancy, and original cached/full-forward logit differences below 0.00005. Float16 storage passed a 24-context / four-model precision check (maximum probability change 0.000740); bfloat16 did not pass the 0.001 tolerance. These checks verify execution, not forecasting quality.
Reproducibility
Setup, scope and downloads
Download compact JSON · Download all epoch metrics (CSV)
Full-label and one-label differ in both supervision and group composition, so their difference is not a clean missing-label ablation. Within each experiment, the two arms share examples, seeds, cached contexts and labels.
What was trained
Qwen3-0.6B, pinned revision 6130ef31402718485ca4d80a6234f70d9a4cf362. The original model generated reasoning and its KV states once; those states stayed frozen. All model parameters were trainable only through the answer suffix \n\n[YES/NO]:. P(YES) is the normalized probability between the single tokens " YES" and " NO". This is supervised suffix training with an immutable original-model context—not RL or training the model to generate new reasoning.
AdamW learning rate 10⁻⁶, zero weight decay, gradient clipping 1, ten groups per step, consistency weight 1, seed 20260917. Cache storage float16; computation float32. Pilot consistency uses squared negation residuals, squared containment hinges, and squared AND/OR-sum residuals. Evaluation uses complements, signed AND/OR identities and containment bounds; it does not cover every possible coherence constraint.
Markets were selected from an archived, filtered Manifold dataset and split by closing week. Training selection used partial relevance-tournament scores and disjoint embedding pairs; topical similarity does not guarantee dependence. Historical resolution criteria were not fully cleared for hindsight leakage. Original-model reasoning used temperature 0.6, top-p 0.95, top-k 20 and a 2,048-token limit; 13 training and 3 validation traces required a forced closing token, and were retained.
The initial one-epoch pilot (38 updates per arm) and five-epoch extension are included in the full-label history, not counted as independent replications. Raw private operational receipts, host paths, model weights, market text and caches are deliberately outside these public downloads. JSON includes source-history hashes for provenance.
How to read the metrics
Log-loss is mean binary cross-entropy in natural-log units: −[y ln p + (1−y) ln(1−p)]. Brier is mean (p−y)², without a factor of two. Lower is better for both and for consistency MSE. Online training BCE averages losses during parameter updates; it is not a separate post-epoch evaluation. Best-checkpoint selection is based on overall validation log-loss, not accuracy or consistency.
At their best log-loss checkpoints, consistency MSE is 0.053608 for the consistency arm versus 0.041875 for accuracy-only: the best forecast-loss checkpoint is not necessarily the most coherent. Best-consistency singles score 0.640502 versus the 0.693147 baseline, a tentative signal requiring a larger untouched evaluation.