Local LLM benchmarks on AMD Strix Halo

The 12B result changes the Gemma lineup. It writes 28/30, one point above the 31B, with all five required beats, distinct character voices, and a clean opening. Postgres is 47/57, third best on the board, with perfect diagnosis and 26/31 on procedural tasks. Cassandra is 38/56, tied with the 26B-A4B and one behind the 31B. Polyglot is 32/65 with a perfect Go rule engine and a partial cron solve at 6/10.

Speed is the only weak point. At Q8 a dense 12B is memory-bandwidth-bound here, so generation stays around 14 to 15 tokens per second across all three backends. ROCm leads prompt processing at 928 against RADV's 716, but generation barely moves. A Q4 build would roughly double throughput at some quality cost, so Q8 stays the quality reference until a lower-precision run proves itself.

The rest of the family still matters. The E2B is the fastest model I have tested at 109 tokens/second and under 3 GiB. The 26B-A4B MoE leads Gemma on Cassandra tie-breaks and posts 49/57 on PostgreSQL. The dense 31B still owns the top combined score at 238/285, but the new 12B is close enough to be the practical default for quality testing.

The dense 27B is the story here. 1.84x token generation at n=3 with zero quality cost, because MTP is lossless draft-and-verify rejection sampling at the target distribution. The server-side baseline of 11.58 tok/s matches the 12.03 llama-bench reference within chat-template overhead. The win on the dense model comes from being memory-bandwidth-bound during decode. The MTP head's extra forward pass is cheap relative to the wait on weight bandwidth, and most drafts get accepted, so the saved verifies are nearly free wall time.

The MoE 35B-A3B gains only 1.22x. With 3B active parameters decode is already compute-light, MTP overhead eats most of the would-be savings, and the absolute baseline of 54.85 tok/s leaves less room to grow. The acceptance rate is still healthy at 71 to 78 percent. The mechanism works fine, the headroom is just smaller. If memory is tight the regular Unsloth GGUF (without the extra 0.42B MTP layer) is the cleaner choice. If the model is loaded anyway, leave MTP on.

Acceptance rate tracks prompt determinism, not model. Both models accept above 80 percent on summarize, code_python, and translation. Both drop below 55 percent on creative_short, which is a four-line poem about a lighthouse. High-entropy generation kills draft acceptance because every plausible continuation breaks the draft. The 27B holds 1.60x speedup even at 54 percent acceptance because the dense model has so much bandwidth slack to absorb the failed drafts. The MoE collapses to 1.05x on the same prompt. Per-prompt best at n=3 on the 27B is summarize at 25.58 tok/s (2.13x), on the 35B-A3B is summarize at 87.79 tok/s (1.48x).

n=3 beats n=2 in aggregate for both models. The wider draft window catches more tokens per accepted cycle, which more than compensates for the lower per-draft acceptance rate. Prompt processing was not measurably degraded at the standard -b 2048 server config across this nine-prompt suite. The PR warns of a D2H embedding-transfer penalty on PP, but the prompts are short enough that PP is not the bottleneck. ROCm with MTP is the open question. The ROCm build was last refreshed before the MTP merge, so a rebuild and rerun is the next step there.

Setup is llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -b 2048 -t 32 -fa 1 -c 8192 --spec-type draft-mtp --spec-draft-n-max 3. RADV throughout, mainline llama.cpp at 4f13cb742 (build 9191). For the 27B this becomes my default writing config going forward. For the 35B-A3B it stays on by default since the model is loaded anyway, but the dense 27B with MTP is now the throughput-quality sweet spot on Strix.

Creative writing is where MTP gives back the least, and that is exactly the workload most Strix Halo users care about. High-entropy prose has the lowest draft acceptance rate because every plausible next token breaks a draft, and the presence_penalty=1.5 that the Qwen3.6 family needs to avoid prose loops further depresses acceptance since the MTP head trained on the unmodified distribution. The 1.37x speedup falls well below the 1.84x peak, but on a 2294-word generation that is the difference between 186 seconds and 254 seconds. Quality held, with all five required beats landed in order and no looping.

Polyglot is where MTP gets out of its own way. Code has predictable continuations like function signatures, common idioms, and well-known boilerplate, so per-challenge draft acceptance lands above 70 percent. The bench scored 43/65 in 4.2 minutes wall time, eight points above the previous Qwen3.6-27B best. T2 Cron solved 10/10, which is unusual since T2 plus T5 Rate Limiter plus T6 Sinatra have been the three universal fails across nearly every model on this bench. The Cron win likely came from the corrected repeat_penalty=1.0 (Qwen3.6 Unsloth default) instead of the prior 1.1 (Qwen3 family default), not from MTP itself. Polyglot has documented single-run variance so 43 carries a one-run asterisk.

For practical Strix Halo use, dense Qwen3.6-27B is the model that benefits most from MTP because gfx1151 decode is memory-bandwidth-bound and the MTP head's extra forward pass is cheap relative to the bandwidth wait. The MoE 35B-A3B sees a smaller 1.22x gain because 3B-active decode is already compute-light and leaves no slack for MTP to absorb. For users whose primary workload is long-form writing or daily-driver coding, the configuration is Qwen3.6-27B + MTP n=3 + the Unsloth Qwen3.5-family sampling profile (presence_penalty=1.5 for writing, repeat_penalty=1.0 across the board). Expect 16 to 21 tok/s depending on workload, with no quality regression.

Combined 207/285 lands the model 5th overall on the leaderboard, behind Gemma-4-31B (238), Gemma-4-12B (222), Gemma-4-26B-A4B (215), and Qwen3.6-27B (213), with Qwen3.6-35B-A3B (205) just below. Writing scores 30/30, joint top with Qwen3-30B-Instruct-2507 and Qwen3.6-27B. The story output runs 2082 words against the 2000-word target, reaches all five required beats, and carries a structural payoff that earns the score on craft rather than padding. The hatchling rescue at the heart of the scene parallels the patient grief introduced in the opening, and the model lands the parallel through dialogue (a flat "wasn't meant to" delivered over a struggling baby turtle) rather than narration. Em dash count stays inside budget, sensory layering covers salt, sand texture, light, and skin warmth without any of the AI-prose tells the bench scorecard treats as deductions. Generation cost was 1090.8 seconds (1.9 words per second on ROCm).

The LRU coding score requires explanation. The standard server bench scored 2/10 because the default no-reasoning path generated a put method that does time.time() + (ttl if ttl is not None else self.default_ttl), which TypeErrors when both arguments are None. Re-running the same prompt with reasoning_effort: high through chat-template-kwargs produced 18,606 characters of reasoning across 28 minutes and a different implementation that scores 7/10. The new failures are all the same conceptual bug in a different place. The eviction path runs del self.expiry[oldest_key] unconditionally, which KeyErrors when the evicted key was stored without an expiry entry. Two attempts, two different bugs, both centered on None handling in the expiry map. The 7/10 with reasoning is the recorded score because that mode is the model's intended high-effort coding profile per the Unsloth docs.

Polyglot scored 30/65 on a single run, which puts the model 3rd on this benchmark behind Gemma-4-31B (44) and the Qwen3.6 pair (35 and 30). T7 Go and T4 CSV both perfect (10/10 and 7/7). T2 cron, T5 rate limiter, T6 Sinatra all 0/10, the same three challenges that have stayed unsolved across nearly every tested model. Polyglot variance on this bench is well-documented (one earlier model produced 6, 17, 19, 26, 35 across five identical runs), so 30/65 carries a single-run asterisk pending re-bench. Postgres came in at 39/57. T3 diagnosis was perfect 6/6, T2 query optimization 9/10, T1 SQL writing 5/10, T4 procedural 19/31. The pattern is consistent. The model fixes broken queries and identifies operational problems with surgical accuracy and writes original complex SQL less reliably. Cassandra 34/56 with T2 anti-pattern detection at 9/10, the strongest non-Gemma result on that tier, plus T1 query writing 8/10. T4 procedural collapsed to 12/30 with broken CQL DDL on time-bucketed schema, materialized view, batch denormalization, and SAI index challenges.

ROCm is the right backend here. pp512 runs 78 on ROCm, 62 on RADV, 17 on AMDVLK. Token generation is essentially identical across all three (2.92, 2.90, 2.99) because at 125B dense the workload is pure memory bandwidth. Build 9029 for Vulkan, 8721 for ROCm.

The reasoning interface matches Mistral-Medium-3.5. The template reads a reasoning_effort kwarg with two settings, none and high, and the bench's default enable_thinking: false is ignored. I set reasoning_effort: high at the server for the coding and database suites, and that choice exposed a sharp edge. The first LRU run scored 0/10 with an empty response. At high effort the model wrote 18,000 characters of reasoning into the separate reasoning_content field and hit the 4096-token cap before a single line of code reached the answer. Raising the limit to 20480 tokens fixed it and LRU recovered to 10/10. Anyone benching this model at high effort needs the larger budget or the code never lands.

Pure coding is the strong suit. 10/10 LRU, a clean 59/59 LeetCode, and 24/65 polyglot, which clears the Gemma 26B MoE and sits in the upper middle of the field. The Go rule engine missed by one line, an unused loop variable that Go treats as a compile error and that zeroed all ten tests. Postgres reached 27/57 with Tier 2 optimization at 9/10 and Tier 3 diagnosis at 5/6, while Tier 1 lost points to careless slips like a numeric typed as numic and an interval compared against an integer. Cassandra came in at 26/56 after I made the JSON extractor lenient. The model identified every Tier 2 anti-pattern correctly but pretty-printed its CQL fixes as multi-line strings with raw newlines, which strict JSON parsing rejected. Switching to strict=False moved Tier 2 from 5 to 7.

Creative writing is the weak spot at 21/30. The story hits all five required beats and the sensory work is good, with sand clinging like ghosts and the gritty cold of the dawn beach. The prose leans hard on telling, naming emotions in italics rather than showing them, and Jack stays flat next to Maya's sharper voice. Two things drag the score below the competent low twenties. The model breaks character at the end with a full assistant sign-off that offers to revise the story, and it prints a fabricated 2,000-word count when the actual text runs 2,364. It also reworks the brief, making Jack's late wife the very patient Maya lost, a coincidence the two-strangers setup never asked for.

RADV is the backend here. It leads token generation at 40 tg and trails ROCm only on prompt processing (363 against 441 pp), and tg is what matters for writing. At 6B active the model decodes more than ten times faster than the dense Mistral-Medium-3.5 at 3 tg, which is the whole point of the MoE. Build 9518 for Vulkan, 9172 for ROCm.

Granite-30B's writing score of 28/30 ties MiniMax-M2.7 for the highest non-Qwen result on the writing bench. Combined 172/285 ties Qwen3-30B-Instruct-2507 exactly. The writing benchmark is a 2000-word creative-writing prompt about two strangers, an ER nurse and a marine biologist, meeting at dawn on a North Carolina beach. Granite-30B hit all five required beats. Maya's wry voice ("Glad I could be your personal sand excavator") and Jack's deflective grief ("Close enough", thumb on the scar at his jaw) stay distinct throughout. Two named griefs land hard. The patient she held through that January night, the late wife implicit in his deflection. Sensory detail layers across mildew and salt, the "gray ribbon" beach, molten gold sunrise. One mild meta closer at the end ("The story ends here, but the question lingers...") cost a quality point. The model also EOS's at 1244 words against the 2000-word target. Both look like training tics rather than capability ceilings, since the same closer pattern shows up at all three sizes.

Granite-4.1-8B is the first model on this benchmark to score a perfect 22/22 on the hallucination calibration test. All ten factual answers correct, all eight false-premise traps caught, all four unanswerable questions answered with appropriate uncertainty, in 12.9 seconds. Qwen3.5-35B-A3B took 76.7 seconds for 21/22. The 8B also scores 59/59 LeetCode at 38.6 tg. Combined 141 sits within margin of Qwen3.5-9B at 146. Granite-4.1-3B (Combined 98) scores 53/59 LeetCode at 88.7 tg, the fastest tg of any sub-100 Combined model, but landed 0/10 on LRU and 0/8 on FastAPI. Both are genuine generation failures. The LRU code stores TTL as a single float in ttl_thresholds[key] and then unpacks it as (value, ttl) in _evict_expired, throwing TypeError on every test. The FastAPI code uses Body({"title": ...}), which is invalid Python syntax, and Body was never imported. Server crashed on startup.

Granite-30B trades throughput for quality. At 11.76 tg the dense model runs roughly five times slower than the Qwen3-30B MoE that ties it on Combined. Postgres T2 query optimization scored a perfect 10/10. FastAPI scored 2/8 across all three sizes, with HTTP 422 on POST and PUT. The Pydantic body schema is the failure mode and sampling does not fix it. The IBM RL push on tool calling did not transfer to FastAPI body schema generation. ROCm wins prompt processing on every size (3b 2323 vs 2278, 8b 1211 vs 936, 30b 303 vs 275). RADV wins token generation everywhere. Pick RADV for writing, ROCm for long-prompt server workloads on the 30b.

Combined 148/285 puts the Omni below the field's working models. Writing landed at 24/30, identical to the older text-only Nemotron-3-Nano-30B-A3B. Multimodal training did not move prose capability. RADV runs 1097 pp / 61.1 tg, beating AMDVLK (777/58.7) and ROCm (879/56.8 on the older b8721 build).

The model needs separate sampling profiles for separate tasks, per the Unsloth docs. Code generation wants Instruct-mode params (temp 0.2, top_k 1). LRU cache jumped from 0/10 to 8/10 with that switch. Default Thinking-mode params (temp 0.6, top_p 0.95) had produced cleanly extracted but buggy code that stored None in the expiry map and then crashed on None <= now. The same Instruct switch dropped polyglot from 6/65 to 2/65 because multi-challenge breadth wants the variance. Running a mixed workload through one llama-server instance means picking one profile and accepting losses on the other side.

FastAPI scored 2/8. The model writes the POST endpoint with a query-string title parameter instead of a Pydantic JSON body, so every POST returns HTTP 422. Sampling does not fix design failures. Cassandra T4 procedural collapsed to 2/30 with syntactically broken CQL DDL across LWT locks, materialized views, counter rate limiters, and batch denormalization. Postgres T2 optimization held at 9/10 and T3 diagnosis at 4/6. Query work is fine, procedural code is not.

Qwen3.6-27B lands 4th place on the Combined leaderboard at 213/285, behind Gemma-4-31B at 238, the new dense Gemma-4-12B at 222, and Gemma-4-26B-A4B at 215. At this size class MoE usually wins, but the 27B dense beats its 35B-A3B MoE sibling at 205. The dense version takes around five times longer per token (12 tg vs 60 tg) in exchange for one writing point and a much stronger Tier 4 procedural Postgres score.

Writing scores 30/30, the joint highest result I've recorded on this benchmark, tying Qwen3-30B-Instruct-2507 as champion. Em dash count is 2 across roughly 2150 words. The same prose hygiene issue from the 35B-A3B sibling shows up here. Even with enable_thinking:false set through chat-template-kwargs, the model leaks a planning outline above the output and an end-thinking tag below it. The generation itself (everything after the planning preamble) is clean and that's what got scored.

Polyglot averages 30/65 across three runs (24, 31, 34). The variance traces almost entirely to T7 Go, which scored 0/10, 10/10, 10/10. T1 regex parsing was perfect 10/10 every pass, T4 CSV aggregation perfect 7/7 every pass. The T2 cron matcher that the 35B-A3B sibling cracked on 4 of 5 runs stayed at 0/10 here. Cassandra averages 32/56 across three runs (30, 32, 33). T1 climbed each pass (6, 7, 8), T3 saturated at 6/6 every run, T4 stuck at exactly 10/30 every run with the same failure signature on LWT distributed lock, materialized view, and batch denormalization. Postgres came in at 44/57 on a single run. T2 and T3 were perfect (10/10 and 6/6), T4 procedural hit 23/31, the second-strongest result on disk after Gemma-4-31B's 22/31.

At the same quant and size as Qwen3.5-35B-A3B, the 3.6 sibling moves Combined from 192 to 205, now 5th overall behind Gemma-4-31B (238), Gemma-4-12B (222), Gemma-4-26B-A4B (215), and the dense Qwen3.6-27B (213). Writing improves one point to 29/30. LRU, LeetCode, FastAPI all max out the same. Polyglot jumps from 17 to 35 on a best-of-5 basis, which needs context. I ran five identical back-to-back polyglot passes and got 19, 26, 6, 17, and 35 out of 65. Same sampling params, same prompts, same model weights. On the 35 run the model cleanly solves cron matching (10/10) and Go rule engines (10/10), two challenges that were previously unsolved across the Qwen 3.5 lineup. On the 6 run it produces almost nothing usable. I suspect the hybrid DeltaNet layers are amplifying sampler noise, though I have not isolated it further.

Both database benchmarks regress a few points relative to the 3.5 generation. Postgres drops 32 to 31, Cassandra drops 38 to 33. The regressions come entirely from Tier 4 procedural challenges where the model generates syntactically broken PL/pgSQL and CQL DDL. T1 through T3 hold up. Postgres T2 and T3 are perfect (10/10 and 6/6). Cassandra T1 through T3 land at 8/10, 9/10, 6/6. The model understands the queries and the diagnosis, but assembling a valid CREATE TRIGGER or CREATE MATERIALIZED VIEW is where it breaks.

The creative writing output has a prose hygiene issue worth flagging. With enable_thinking:false set through chat-template-kwargs, the model produces a clean 2260-word story that I scored 29/30 (all five beats hit, 5/5 sensory immersion), and then dumps its own internal planning notes after the final sentence. Self-correction remarks, word-count checks, structural bullet points. The story itself is clean. The raw generation is not directly usable without post-processing. I think this pattern is embedded in training rather than fixable through sampling params alone.

The 35B-A3B MoE is the standout for speed and creative writing. At 60 t/s it runs 5x faster than the dense 27B while scoring higher on writing (28/30 vs 25/30). All models require Unsloth's recommended sampling params or they produce garbage. The 122B-A10B leads the family on PostgreSQL (36/57), Polyglot (13/65), and writing (29/30), but at 21 t/s it's 3x slower than the 35B MoE. On Cassandra the 122B improved to 37/56 via the server API, close to the 35B's 38/56. The 9B is the only model in either family to solve PostgreSQL recursive cycle detection. RLS defeats everyone.