## Mammoth-1 leads the decision-model sample

Mammoth-1 answered 1139/1342 correctly (84.9%), twelve more than Span-02 and Drex Max, which tied at 1127/1342 (84.0%). GPT-6 Sol without reasoning answered 1143 correctly (85.2%), four more than Mammoth. The four-answer gap does not establish a reliable general ordering. Mammoth’s median response was 240 ms and Sol’s was 1023 ms: about 4.3 times longer for Sol on these measured routes. Sonnet’s separately constrained, no-thinking reference remains 1070/1342 (79.7%).

## The leader still has specific weak operations

Mammoth scored 64/67 on one-to-one assignments—matching each person or object to exactly one available slot—and 57/58 on exceptions and priority rules. Tracking what others know gave 47/50. These tasks can ask where someone will search after an object moved while they were absent. Counting exactly true claims gave 44/71, truth-tellers and liars 48/74, and ordered procedures only 18/48. Procedures require applying each edit to the current state, not merely recognizing a plausible final answer. Span scored 33/48 there, despite its lower overall total. Mammoth also scored 38/52 on tracking swaps, versus Span’s 28/52.

## Drex Max ties Span overall, with different strengths

Drex Max scored 65/67 on one-to-one assignments and 45/46 on if-then rules, versus Span’s 63/67 and 32/46. On procedures, Drex scored 18/48 and Span 33/48; on worst-case choices without supplied probabilities, Drex scored 5/8 and Span 8/8. Equal overall totals do not mean the same decisions will be correct. Drex’s median/p95 was 286/445 ms, versus Span’s previously measured 206/2157 ms: Span had the shorter median, Drex the shorter tail. Drex’s complete API-reported cost was $0.02842302, normalized to $0.02118 per 1000 questions. Span’s recorded $0.00474628 total was about six times lower.

## Elephant-1 is fast, but not a uniform improvement over Jev

Elephant-1 scored 1004/1342 (74.8%), below Jev 1.14’s 1030/1342 (76.8%). Its 139 ms median was shorter than Jev’s 179 ms. Elephant still scored 60/67 on one-to-one assignments and 46/50 on knowledge tracking, versus Jev’s 57/67 and 44/50. Comparing time intervals gave Elephant 25/42 and Jev 40/42; checking required conditions gave 38/52 and 48/52. A newer release is not automatically stronger across this bank.

## The added questions expose more separation

On the original 1136 questions, Mammoth scored 1019/1136 (89.7%), Drex Max 1004/1136 (88.4%) and Elephant 902/1136 (79.4%). On the 206 additions their scores were 120/206 (58.3%), 123/206 (59.7%) and 102/206 (49.5%). This is a descriptive split, not a second leaderboard or a change in weighting. Across the six new categories, Mammoth scored 30/48, Drex 29/48 and Elephant 22/48. Each new category contains only eight questions; one answer moves its score by 12.5 percentage points.

## Jev’s position is a measured result, not an explanation of its roadmap

Mammoth answered 109 more questions correctly than Jev 1.14; Drex Max answered 97 more. Jev still answered all 46 if-then questions correctly, versus 45 each for Mammoth and Drex. This bank measures supplied-fact, five-choice decisions without tools or agentic iteration; it does not test broad knowledge, multilingual coverage or product integration. It cannot establish why a developer has or has not improved a model.

## Free launch access is separate from a production price

Mammoth and Elephant used Baseten’s direct API at the captured promotional zero tariff. Their responses did not contain monetary receipts, so the report does not invent an API-reported $0 price or plot one on the logarithmic cost axis. Baseten advertises a one-week launch promotion; availability and later prices may change. Their 20-question OpenRouter controls agreed with the direct route on all 20 choices for each model. Those controls are not pooled into the full-bank scores or latency measurements.

| Model | Type | Correct | Accuracy | Median | p95 | Cost/1000 | Actual cost |
| --- | --- | --- | --- | --- | --- | --- | --- |
| GPT-6 Sol — no reasoning | LLM reference | 1143/1342 | 85.2% | 1023 ms 🟥 Slow | 1546 ms | $0.632112 | $0.848294 |
| Mammoth-1 | Decision model | 1139/1342 | 84.9% | 240 ms 🟩 Fastest | 666 ms | Promotional free access* | Promotional free access* |
| Drex Max | Decision model | 1127/1342 | 84.0% | 286 ms 🟨 Fast | 445 ms | $0.021180 | $0.028423 |
| Respan Span-02 | Decision model | 1127/1342 | 84.0% | 206 ms 🟩 Fastest | 2157 ms | $0.003537 | $0.004746 |
| Perplexity Decider v1.2 | Decision model | 1120/1342 | 83.5% | 249 ms 🟩 Fastest | 332 ms | $0.007000 + ? | $0.009394 + ? |
| Nimble v4 Max | Decision model | 1115/1342 | 83.1% | 235 ms 🟩 Fastest | 329 ms | $0.027633 | ~$0.037084 |
| d3-turbo | Decision model | 1106/1342 | 82.4% | 165 ms 🟩 Fastest | 298 ms | Free public access* | Free public access* |
| Liquid d1-ultra | Decision model | 1084/1342 | 80.8% | 200 ms 🟩 Fastest | 393 ms | $0.012124 + ? | $0.016270 + ? |
| Amos 1 Large | Decision model | 1074/1342 | 80.0% | 311 ms 🟨 Fast | 360 ms | Free public access* | Free public access* |
| Sonnet 5.5 — no thinking, JSON-choice | LLM reference | 1070/1342 | 79.7% | 1506 ms 🟥 Slow | 2424 ms | $1.433613 | $1.923908 |
| kettei-fat | Decision model | 1066/1342 | 79.4% | 174 ms 🟩 Fastest | 214 ms | $0.015433 | $0.020711 |
| Jev 1.14 | Decision model | 1030/1342 | 76.8% | 179 ms 🟩 Fastest | 269 ms | $0.024571 | $0.032974 |
| Jev 1.13 | Decision model | 1026/1342 | 76.5% | 189 ms 🟩 Fastest | 323 ms | $0.024571 | $0.032974 |
| Mercury Decide | Decision model | 1017/1342 | 75.8% | 238 ms 🟩 Fastest | 388 ms | $0.006956 | $0.009334 |
| Elephant-1 | Decision model | 1004/1342 | 74.8% | 139 ms 🟩 Fastest | 272 ms | Promotional free access* | Promotional free access* |
| Qwen3.8-Max — none, Hyper Charm | LLM reference | 967/1342 | 72.1% | 1191 ms 🟥 Slow | 1731 ms | $0.614821 + ? | $0.825090 + ? quota |
| Liquid d1 | Decision model | 965/1342 | 71.9% | 206 ms 🟩 Fastest | 827 ms | $0.005773 | $0.007748 |
| Cloudflare Clef | Decision model | 936/1342 | 69.7% | 268 ms 🟨 Fast | 417 ms | $0.017594 | $0.023611 |
| Perplexity Decider Flash v1.2 | Decision model | 923/1342 | 68.8% | 159 ms 🟩 Fastest | 231 ms | $0.003406 + ? | $0.004570 + ? |
| Microsoft-Decision-1 | Decision model | 921/1342 | 68.6% | 283 ms 🟨 Fast | 566 ms | $0.012237 | $0.016422 |
| Drex v2.0 | Decision model | 915/1342 | 68.2% | 232 ms 🟩 Fastest | 755 ms | $0.016661 + ? | $0.022359 + ? |
| Cloudflare Clef Flash | Decision model | 860/1342 | 64.1% | 245 ms 🟩 Fastest | 353 ms | $0.008797 | $0.011806 |
| Cloudflare Clef Omni | Decision model | 860/1342 | 64.1% | 241 ms 🟩 Fastest | 400 ms | $0.061511 + ? | $0.082547 + ? |
| Drex v1.5 | Decision model | 853/1342 | 63.6% | 203 ms 🟩 Fastest | 649 ms | $0.011160 | $0.014976 |
| Kev 4B | Decision model | 822/1342 | 61.3% | 225 ms 🟩 Fastest | 377 ms | $0.011718 | $0.015725 |
| Luna 6 Decisions | Decision model | 758/1342 | 56.5% | 254 ms 🟨 Fast | 527 ms | $0.035840 + ? | $0.048097 + ? |
| Tev1 4B experimental (OpenRouter native adapter) | Decision model | 751/1342 | 56.0% | 198 ms 🟩 Fastest | 253 ms | $0.015657 | $0.021011 |
| ScaleDown Classify-1 | Decision model | 550/1342 | 41.0% | 268 ms 🟨 Fast | 362 ms | $0.021388 | $0.028703 |
| Solar Decide Flash | Decision model | 538/1342 | 40.1% | 540 ms 🟧 Moderate | 747 ms | $0.030203 + ? | $0.040532 + ? |


The comparison contains 26 specialized decision models and three no-reasoning LLM reference routes: 29 × 1342 = 38,918 first attempts. All earlier 26 model records are preserved exactly. New full-bank runs: October 21, 2026, America/Toronto; Mammoth and Elephant direct through Baseten, Drex Max through OpenRouter’s pinned DeepInfra endpoint.

Same frozen 1342-question bank, seed 2026101906, original five choices and native inputs. Every first attempt remains in the accuracy and latency denominator, including failures. One recovery is allowed only for transport failures or HTTP 408/429/5xx; no answer-dependent retries. All 4026 new first attempts returned usable decisions, with zero recoveries and zero probability-format warnings. After at least 20 questions, a cumulative median above 2000 ms stops a route; none of the three reached that threshold.

Response timing starts before a fresh HTTPS connection and ends after the full response body. Preparation, deliberate pacing, queue waits and recovery delays are outside the response timer. Two requests per model run in barrier waves, with up to three models active. Provider hardware, dates, loads and routes are not controlled away; latency is not batch throughput or an isolated inference-speed measurement.

Baseten’s catalog and returned latest serving aliases identify Mammoth-1 and Elephant-1, not immutable weights. Drex returned nace-ai/drex-max-20261016 from DeepInfra. Baseten’s official technical report describes one-pass option-logit decisions. These native requests ask for neither reasoning nor explanations. Reported native output-token counters are retained; a nonzero counter alone is not evidence of thinking. Undisclosed internal execution cannot be independently verified from an API envelope.

Drex uses reported usage.cost for every attempt. Baseten promotional access is labelled separately from metered receipts. Separate compatibility probes and the two 20-case OpenRouter controls are excluded from the 4026 ranked first attempts. Missing charges remain unknown, not zero. Existing Nimble price estimates and Qwen prepaid-quota valuations retain their original labels.

This is a developing benchmark with uneven category sizes and dependent variants, not 1342 independent observations or a pristine held-out evaluation. The 37 permanent author-development exclusions retain their separate 1305-case diagnostic. Two audited ambiguous original items were already replaced before these runs. All 206 additions retain their existing independent verification and computational evidence; the original bank received a targeted difficult-item audit, not independent external verification of every original question.

Public exports contain unchanged answer-free question inputs, aggregate category scores and sanitized evidence hashes—not answer keys, per-item correctness, private verifier outputs, raw account state or credentials. Public checks verify consistency but cannot reproduce private-key scoring or raw-latency pooling. REAX 1 is not included: a hosted API for that exact fine-tune has not been confirmed.