How Well Does Qwen3.8-27B Run on the Intel Arc Pro B70?

Update — August 2026: I kept working on the vLLM XPU setup after publishing this post, and the final configuration I am actually using now is different from the initial benchmark-only setup. The important changes are that prefix caching is enabled, vision input is enabled, and the OpenAI-compatible server is started with Qwen tool-calling and reasoning parsers for Pi. In that configuration, I measured about 424.8 t/s prefill on a cold repeated prompt and about 1388.7 t/s on the warm repeat, while decode stayed in the low-to-mid 80 t/s range. So the current recommendation is still vLLM XPU for single-stream Qwen3.8-27B, but specifically with caching enabled and the full agent-facing configuration, not the stripped-down benchmark launch.

Qwen3.8-27B just dropped, and I wanted to see how well it performs on the Intel Arc Pro B70 and what the best output we can get is, specifically with MTP enabled.

I did not know that you could get a dense model this large running at this speed. Getting this thing to 84.56 t/s makes it absolutely not only viable, but more than likely the better choice for most workloads and a model worth upgrading to after all.

I’ve already done this same set of benchmarks against Qwen3.6-35B-A3B, the MoE version. That model has been my daily driver for fast, snappy agentic workloads where I can fire off a bunch of things in parallel and let it go get work done. Qwen3.8-27B is a little bit different. This is a dense model, and I expected it to run substantially slower on this card. That was true through llama.cpp. It was not true once I got vLLM XPU and MTP4 working.

Granted, I did have to apply a specific patch before starting vLLM to keep the preserved MTP draft layers in BF16. So this was not something that worked out of the box, but once the patch was applied, the performance changed substantially.

The official Qwen3.8-27B results show a substantial jump from Qwen3.6-27B to Qwen3.8-27B, especially on coding, software engineering, tool use, and agentic tasks. Some of the results even put it close to or above Opus4.6 Max on specific benchmarks. Those results are not the same thing as testing the model on my own work, but they are the reason I wanted to get it running and see what it could do on the B70.

First, though, we have to test the performance.

The short answer

Qwen3.8-27B runs amazing on a single B70 through vLLM XPU, and MTP improves the overall performance substantially. 🤯

The fastest result I measured was through vLLM XPU with a GPTQ Int4 checkpoint, MTP4, and our own audited BF16 draft patch. On the p512/g128 single-request test cell, it reached a median client-side post-first-token decode speed of 84.56 t/s. I did not figure out this vLLM configuration on my own. I found it through another B70 owner’s published findings, then audited and reproduced the required patch and validated the result on my own hardware. I’ll get into that process later in the post.

In contrast, the best llama.cpp configuration I measured was SYCL with embedded MTP enabled at draft depth 2:

--spec-type draft-mtp --spec-draft-n-max 2

That configuration reached a meager 28.40 t/s, compared with a 21.41 t/s SYCL baseline without MTP. Vulkan improved even more in percentage terms, moving from 16.02 t/s to 26.80 t/s. This matched what I saw in my previous dense Qwen3.6-27B testing and was almost too slow to be usable.

The winning draft depth was the same on both backends: n=2.

The vLLM result actually puts Qwen3.8-27B above the 76 t/s single-stream decode result from my previous Qwen3.6-35B-A3B test. Given the initial quality results, I don’t know how I couldn’t completely switch over at this point, provided a little bit more testing validates what I have seen so far.

There is one major limitation. This is a single-stream recommendation. Two overlapping requests with MTP enabled can crash the current XPU engine, and I experienced that myself. If you need multiple streams, you need to turn MTP off for now or wait for that path to be fixed upstream.

Why I expected the dense model to run slower than the MoE

My previous understanding was that dense models should run substantially slower because every token has to work through the entire model. If the quantized model is around 17 GB, the basic expectation is that those weights have to be read for every generated token.

My understanding of the MoE configuration was different. Qwen3.6-35B-A3B is a much larger model on paper, but it only activates a small number of parameters for each token. In my previous B70 benchmarks, the Qwen3.6 MoE reached 76 t/s, while the dense Qwen3.6-27B model landed around 20 t/s.

That is a major difference.

My working theory was that this wasn’t necessarily because the dense model was badly optimized. It was because the B70 had to move substantially more weight data for every generated token. The B70 is a 32 GB card with 608 GB/s of memory bandwidth, so I expected the sparse MoE architecture to have a major advantage on this hardware.

So I did not expect Qwen3.8-27B to replace the Qwen3.6 MoE as the fastest model on the card. The purpose of this test was to see how close we could get, whether MTP could help, and whether the quality was good enough to justify the dense workload.

The vLLM result changed the practical conclusion. It does not necessarily disprove the underlying dense-versus-MoE theory, but it does show that the runtime and MTP implementation can completely change what that difference looks like in actual use. With perfect acceptance on this test cell, the final single-stream result was faster than I thought a dense model of this size could run on the B70.

If my previous understanding still holds, then a future Qwen3.8 MoE running through vLLM XPU with MTP enabled should theoretically run even faster. That is my expectation, not a result. We do not have that model or the benchmarks yet, so it remains to be determined.

The llama.cpp benchmark setup

Back to the llama.cpp benchmarks for those still interested…

I used the unsloth/Qwen3.8-27B-GGUF release with Qwen3.8-27B-UD-Q4_K_XL.gguf on a single Intel Arc Pro B70.

The common configuration included:

• Full GPU offload with -ngl 999
• Flash attention enabled
• q4_0 K and V cache
• 131072 context
• --parallel 1 for the MTP comparison
• Fresh Vulkan and SYCL llama.cpp builds

For MTP, I tested the baseline with MTP off, followed by draft depths 1 through 4:

--spec-type draft-mtp --spec-draft-n-max 1
--spec-type draft-mtp --spec-draft-n-max 2
--spec-type draft-mtp --spec-draft-n-max 3
--spec-type draft-mtp --spec-draft-n-max 4

Both backends logged that they were creating the MTP draft context against the target GGUF. In this specific Unsloth release and test, the required MTP data was available through the target GGUF and no separate sidecar model was required.

The normal upgrade did not change much

Before getting into MTP, I went back and reran the dense Qwen3.8-27B numbers after upgrading the stack and rebuilding the Vulkan and SYCL binaries.

The normal post-upgrade rerun was mostly steady:

Vulkan stayed flat in single-stream and ticked up slightly in four-way aggregate throughput. SYCL stayed the stronger single-stream backend for this dense model.

So the normal upgrade was not the story. MTP was.

MTP finally improved performance on the B70

Both Vulkan and SYCL have not handled MTP well in my previous benchmarks. MTP is supposed to improve performance by predicting a few possible tokens ahead and then checking those predictions together. In my previous tests, the extra work required to make and check those predictions cost more time than it saved. MTP technically worked, but the final result was slower.

Since then, the llama.cpp and Intel userspace stack changed. The post-upgrade build exposed working integrated MTP on both Vulkan and SYCL, but this benchmark did not isolate the exact contribution of each llama.cpp, Mesa, or Intel runtime change.

In plain terms: I can show that MTP performed substantially better after the upgrade. I cannot honestly credit that gain to one specific upstream change from this test alone.

This time, MTP did perform.

Vulkan MTP results

On Vulkan, turning on MTP immediately changed the picture. n=1 jumped to 24.21 t/s, n=2 peaked at 26.80 t/s, n=3 held close at 26.44 t/s, and n=4 dropped back to 22.41 t/s as acceptance fell off.

The best Vulkan setting was --spec-draft-n-max 2, good for roughly a 67% improvement over baseline.

SYCL MTP results

SYCL started from a much stronger baseline at 21.41 t/s, and MTP still helped. n=1 reached 25.18 t/s, n=2 climbed to 28.40 t/s, n=3 fell back to 24.04 t/s, and n=4 nearly collapsed to baseline at 21.95 t/s.

Again, n=2 was the sweet spot.

The pattern was the same on both backends: shallow draft depths helped, while deeper ones over-proposed. Acceptance started high enough at n=1 and n=2 to pay for verification, then dropped hard at n=3 and n=4.

In other words, the B70 can absolutely benefit from MTP on a dense Qwen model, but only if the draft depth is kept conservative… now back to the exciting stuff!

vLLM XPU pushed Qwen3.8-27B to 84.56 t/s

The 28.40 t/s SYCL result is the fastest result from this particular llama.cpp Vulkan and SYCL benchmark matrix. It is not the fastest result I was able to get from Qwen3.8-27B on the B70.

I was probably 90% finished writing this post when I stumbled across the vLLM findings. I had tried to get vLLM running on this machine before, and it absolutely would not work for a variety of reasons. This time, I was able to get a setup script working, get the server running consistently, and reproduce the substantially faster results.

Another B70 owner originally reported around 53–55 t/s using vLLM XPU and MTP3. They later published a more complete Qwen3.8-27B reproduction guide showing 83.7 t/s with MTP4 on a p512/g128 test.

I wanted to know if those numbers were real. I also didn’t know exactly what was inside the provided patch, so instead of blindly downloading it and running it on my machine, we audited what it changed and reproduced that behavior in our own patch. I then built the vLLM XPU configuration and tested it myself on August 18, 2026.

The vLLM XPU configuration

Here is the environment I tested:

• Backend: vLLM XPU on one Intel Arc Pro B70
• Image: vllm/vllm-openai-xpu@sha256:f01e24f6c7ff01f1e0662234255a1372297d1dbd89d003cf13c8fad3eab1ba4f
• vLLM: 0.27.2rc1.dev77+gac7509e2b.xpu
• Model: SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
• Served model name: qwen38
• Context: 131072
• KV cache: fp8
• GPU memory utilization: 0.88
• Maximum sequences: 64
• Maximum batched tokens: 8192
• Prefix caching disabled
• XPU graph mode enabled
• MTP4 enabled

The server was configured with --max-num-seqs 64, but the headline benchmark kept one active request. Given the current mixed-request crash, I would cap an MTP4 endpoint at --max-num-seqs 1 rather than allowing vLLM to schedule overlapping requests.

The MTP and XPU environment configuration was:

B70_MTP_BF16_DRAFT=1
VLLM_XPU_ENABLE_XPU_GRAPH=1
ZE_FLAT_DEVICE_HIERARCHY=COMPOSITE
ZE_AFFINITY_MASK=0
PYTORCH_ALLOC_CONF=expandable_segments:True

The normal model artifact already contained the MTP data, so no separate sidecar model was needed.

I applied our own audited BF16 draft patch. Its purpose was to force the preserved MTP draft layers to stay unquantized BF16. The logs confirmed that the patched path was active:

[B70] MTP draft: using unquantized BF16 layers

There is a related vLLM pull request for Qwen3.5-family GPTQ and AWQ models with BF16 MTP layers. It addresses the same basic problem by keeping the main model quantized while constructing the MTP draft layers without the target model’s quantization configuration. As of August 18, 2026, it is still open and is not the exact environment-gated patch I ran. For now, you either need to wait for upstream support or hand-roll and audit the change like we did.

I did not apply the separate boundary patch. Its exact source anchor did not match this XPU build, and I wasn’t going to force a patch into a different section of code and hope for the best. That means the BF16 draft MTP path is validated, but exact max-context boundary behavior at 131072 is not fully validated yet.

The final vLLM MTP4 result

The primary test used a 512-token prompt, generated 128 tokens, and kept one active request. I ran a warmup followed by five measured runs using client-side post-first-token decode timing.

The five measured decode results were extremely consistent:

84.78 t/s
84.40 t/s
84.56 t/s
84.50 t/s
84.63 t/s

The vLLM logs also showed why MTP4 worked so well on this test. The mean acceptance length was 5.00. It accepted all 564 drafted tokens, with a 100% average draft acceptance rate and 1.000 acceptance at every speculative position. With four speculative tokens configured, 5.00 is the maximum mean acceptance length: the four accepted draft tokens plus the normal verified token, based on how vLLM defines this metric.

Should we be concerned about 100% acceptance? Not by itself. Speculative decoding still verifies drafted tokens against the full model, so a high acceptance rate does not bypass the target model or lower the quality of the result. Official vLLM reports do show occasional 100% acceptance windows, although rates across broader workloads normally move around.

So on this specific benchmark cell, MTP acceptance was effectively perfect. That’s why throughput jumped into the 84 t/s range. I would still treat 100% as an unusually favorable, prompt-specific result rather than something to expect on every request. I need to test acceptance across a more varied mix of coding, chat, and long-context prompts before making a broader claim.

Before applying the BF16 draft patch, the same direct streaming harness measured 32.75 t/s with MTP off, 45.88 t/s with MTP1, and 55.92 t/s with MTP2. Those earlier results are useful directionally, but they are not a clean mode-to-mode comparison with the final patched MTP4 result.

I also ran the built-in vllm bench serve harness without MTP. It produced 148.62 t/s of aggregate output throughput at 1.16 requests per second, with a median TTFT of 3107.55 ms and median TPOT of 43.34 ms. That harness was not reliable for MTP on this configuration because the MTP path hit a spec/non-spec invocation issue. For the meaningful MTP number, I used the direct streaming, single-request harness.

MTP4 is a single-stream recommendation right now

This setup requires MTP4 to reach 84.56 t/s. Right now, if two requests overlap while one is using the speculative MTP path and the other is starting or processing its prompt, the XPU engine can crash. I experienced that once while using it.

This is tracked in the vLLM XPU kernels concurrency issue. As of August 18, 2026, the mixed-request case is still open and there is no linked pull request that fully fixes it. A partial workaround handled two requests in one test but failed at four, so I would not build a multi-user deployment around that workaround.

If you need the 84.56 t/s result, keep this as a single-stream endpoint. If you need multiple active requests, turn MTP off for now. My direct streaming result with MTP disabled was 32.75 t/s; the 55.92 t/s result used MTP2.

So the measured and defendable result is specific:

Qwen3.8-27B GPTQ Int4 running through vLLM XPU with MTP4 and our audited BF16 draft patch reached 84.56 t/s median client-side post-first-token decode speed on the p512/g128, single-request test cell.

This does not mean every request will run at 84.56 t/s. It does not claim exact 131072 boundary correctness, and it does not validate concurrent MTP serving. It does show that the other published B70 headline cell was independently reproducible on my B70 using our own audited version of the required BF16 draft change. In fact, this run came in slightly faster than the 83.7 t/s result I was trying to reproduce.

Getting this performance is not a matter of changing one llama.cpp flag. It means moving from GGUF on llama.cpp to vLLM XPU with a GPTQ checkpoint and a BF16 draft patch.

If you want the fastest single-request configuration I tested, this is now the recommendation:

vLLM XPU
Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
fp8 KV cache
131072 configured context
MTP4
BF16 draft patch
--max-num-seqs 1

If you want to stay on llama.cpp and GGUF, the best configuration I tested remains:

SYCL
Qwen3.8-27B-UD-Q4_K_XL.gguf
q4_0 K/V cache
131072 context
--parallel 1
--spec-type draft-mtp
--spec-draft-n-max 2

Choosing and tuning inference stacks for production agent workloads is a chunk of what I do as a fractional CTO. Send me an email if this is something you are interested in implementing into your business.

My first quality test was better than I expected

Performance is only the first part of this test. Secondary to the performance, we’re going to have to test how well Qwen3.8 actually does.

My instantaneous results have been really good.

I spun up a quick website through Open WebUI and gave it a very generic “build me a website” prompt, which is one I use pretty often. It went above and beyond specifically on the design and UI side. It used a modern paper style, looked very clean, and felt complete. It did a really good job end to end following through on the task without any additional direction.

This was with thinking off. It essentially one-shot the task and produced a really good output.

Granted, I haven’t reviewed the underlying code closely. I’m making an assumption that the code is nice based on the output, but I haven’t dug deeply enough into its structure to make that claim. We need specific evals and repeatable tests before saying whether it is truly worth switching production software-engineering work over to Qwen3.8.

I’m going to add a couple of the prompts and outputs to this post as well. The initial quality difference is almost incomprehensible. It is so much better that, given a little more testing to validate these results, I don’t know how I couldn’t completely switch over at this point.

The official benchmark results are still difficult to ignore. Qwen reports major improvements over Qwen3.6-27B on Terminal Bench 2.1, SWE-bench Pro, QwenSWEBench, DeepSWE 1.1, OSWorld-Verified, and application recreation. On some of the benchmarks where Qwen publishes an Opus4.6 Max comparison, Qwen3.8-27B is close to or ahead of it. On others, Opus4.6 Max still leads.

That doesn’t prove how the model will perform on my projects. But it does line up with the quality jump I saw in that first test.

Does Qwen3.8-27B replace my Qwen3.6 MoE?

I’m still going to have to figure this out through personal testing. But right now, because of the initial quality results and because the model can run at 84.56 t/s, Qwen3.8-27B is more than likely the better choice for most of my workloads.

That is a substantial change from where I started. I expected to keep Qwen3.6-35B-A3B as my fast, snappy daily driver and only use Qwen3.8 when I needed additional intelligence. At this speed, I don’t know if that split still makes sense.

The exception is parallel work. If you want to fire off several tasks at the same time, the current vLLM XPU MTP path is not ready for that workload. You can run Qwen3.8 with MTP disabled, or you can continue using the Qwen3.6 MoE for those parallel agent loops until the mixed-request problem is fixed.

For a single stream, though, the old tradeoff has substantially changed. Qwen3.8 now gives me both the extra intelligence and more single-stream speed than I expected to get from a dense model this large.

What I’d take from this

My initial impression is that Qwen3.8-27B is absolutely a model worth running. The vLLM result completely changes my view of how well it fits the B70. At 84.56 t/s on this single-request test, the dense model is not just usable on this card. It is more than likely the better choice for most workloads.

The earlier vLLM no-spec streaming result was 32.75 t/s, MTP2 reached 55.92 t/s, and the final patched MTP4 result reached 84.56 t/s. Those runs were not a clean patched A/B comparison, but even the intermediate MTP result was substantially fast and usable. The final MTP4 result changes the entire conclusion of this post.

If you need the highest quality and the best single-stream speed-to-quality ratio I have found for Qwen3.8-27B on the B70, this is the configuration I would use right now. I still need more personal testing before calling the model my permanent daily driver, but I am no longer looking at it as the slower model I would only use occasionally.

The next step is to set up specific coding and agentic evals, add the prompts and outputs, and validate how it performs on real projects. I also need to test longer contexts and exact max-context boundary behavior. Concurrent MTP requests will have to wait for the XPU issue to be fixed.

In the future, all of my benchmarks will have to include vLLM instead of only testing Vulkan, SYCL, or llama.cpp on the backend. vLLM is a top-tier player for sure. In this case, it is the absolute standout and the best of the bunch, with or without the final patch.

This also leaves me very excited about a future Qwen3.8 MoE model. If a dense 27B model can reach this speed on the B70, a Qwen3.8 MoE could theoretically be even faster. That might blow my mind.

For now, if you want maximum validated single-request speed on one B70, the headline configuration is vLLM XPU, GPTQ Int4, MTP4, and the audited BF16 draft patch. If you’re running Qwen3.8-27B through llama.cpp, start here:

--spec-type draft-mtp --spec-draft-n-max 2

That was the winning llama.cpp flag on both backends, and SYCL produced the best llama.cpp result at 28.40 t/s.

If you’re standing up local AI on hardware you own and you’d rather not learn this by finding the right runtime and patch after the post is already 90% written, the architecture and operating discipline around a moving stack is the kind of thing I work through with companies as a fractional CTO. The hardware is one piece of it; knowing what to re-test and when is the other. Send me an email if this is something you are interested in implementing into your business.

FAQ

How fast did Qwen3.8-27B run on the Intel Arc Pro B70?

The fastest result I measured was through vLLM XPU with a GPTQ Int4 checkpoint, MTP4, and our own audited BF16 draft patch. On the p512/g128 single-request test cell, it reached a median client-side post-first-token decode speed of 84.56 t/s.

What is the final vLLM configuration you are actually using for Qwen3.8-27B?

The configuration I am actually using now is the Intel OpenAI XPU vLLM image with MTP4, the audited BF16 draft patch, prefix caching enabled, multimodal input enabled, and Qwen tool-calling support enabled for Pi.

The important runtime pieces are:

--quantization gptq
--dtype float16
--max-model-len 131072
--gpu-memory-utilization 0.88
--kv-cache-dtype fp8
--max-num-seqs 64
--max-num-batched-tokens 8192
--served-model-name qwen38
--speculative-config '{"method":"mtp","num_speculative_tokens":4}'
--enable-auto-tool-choice
--tool-call-parser qwen3_xml
--reasoning-parser qwen3
--limit-mm-per-prompt '{"image":10}'

I am not using --language-model-only, because I want image support enabled, and I am not using --no-enable-prefix-caching, because the warm-cache behavior is materially better for repeated prompts.

What prefill speed are you seeing from the final vLLM setup?

On the current cached, multimodal, tool-calling vLLM configuration, I measured about 424.8 t/s prefill on the cold request and about 1388.7 t/s on the warm repeated request, with decode at roughly 82.7 to 86.1 t/s on that same test path.

What was the best llama.cpp setting?

The best llama.cpp configuration I measured was SYCL with embedded MTP enabled at draft depth 2: --spec-type draft-mtp --spec-draft-n-max 2. That configuration reached 28.40 t/s, compared with a 21.41 t/s SYCL baseline without MTP. Vulkan improved from 16.02 t/s to 26.80 t/s, and the winning draft depth was the same on both backends: n=2.

Is the vLLM XPU MTP4 result a multi-stream recommendation?

No. This is a single-stream recommendation. Two overlapping requests with MTP enabled can crash the current XPU engine, and I experienced that myself. If you need multiple streams, you need to turn MTP off for now or wait for that path to be fixed upstream.

Does this change the dense-versus-MoE conclusion on the B70?

The vLLM result changed the practical conclusion. It does not necessarily disprove the underlying dense-versus-MoE theory, but it does show that the runtime and MTP implementation can completely change what that difference looks like in actual use.

Does Qwen3.8-27B replace Qwen3.6-35B-A3B for your workloads?

I’m still going to have to figure this out through personal testing. But right now, because of the initial quality results and because the model can run at 84.56 t/s, Qwen3.8-27B is more than likely the better choice for most of my workloads.