consistency-ft · corrected comparison details

Model ranking · all 25 model / reasoning settings

GLM-5.3, Grok 4.6, GPT-6 Astra, and GPT-5.6 Luna at none, low, medium, high, xhigh, and max are included alongside every historical model. Scores measure probability consistency, not accuracy. 148 shared tuples across 24 participating settings, from 157 reviewed tuples. All native settings and API runs with complete original coverage participate; incomplete historical settings remain visible and unranked. Selection by shared availability can change the ordering; these results do not represent the full original 300-tuple benchmark. Native subscription runs and historical API runs differ in dates, prompts and reasoning settings. Coverage and methods →

Jev uses a distinct probability channel: TypeSafe Noul returns a direct yes-probability for each question, independently, with its full resolution criteria. No shared question context, logical hints, prompt tuning, verbal answers or token logprobs. Reasoning controls and an explicit abstention field are not exposed. The same algebraic formulas and shared tuple IDs are used; this does not isolate model quality from elicitation method. Research and protocol →

*Historical API runs requested none, with low as a fallback; the setting actually used was not recorded per proposition. Subscription runs were measured in September and historical API runs in August, with different native prompts, reasoning settings, and output limits.

Coverage and comparison limits

Reviewed target: 157 whole tuples across 10 checks, using 274 distinct propositions. The review applied the documented topic and semantic criteria to source texts, without consulting model responses. The ranking uses only the 148 tuples scored by every one of the 24 participating settings. The table below retains each model’s coverage on the full reviewed target, including questions outside that shared set. Missing readouts and incomplete archived coverage can select easier questions; a shared denominator does not remove this selection effect. Reviewed scope and exclusion reasons. Native protocol and audit. Unanswered probabilities remain missing, never zero. Low violation measures coherence, not forecasting accuracy or calibration.

Subscription usage

Across all archived campaigns: 61,738,874 recorded native tokens and 8,753 CLI launches, including setup and interruptions. 76 interrupted launches have no native receipt; native usage is unknown for 106 launches in total. Unknown consumption is not zero. Imported observations are counted once. Quota percentages are not summed across campaigns or accounts.

Quota changes below cover native-20260913-scoped-final: 2026-09-13T04:30:48.606308+00:00 to 2026-09-13T04:55:52.120664+00:00. Account-wide deltas can include other fleet sessions; window resets and provider accounting prevent inference from tokens alone. Download complete usage report.