Is one DGX Spark enough? What it actually runs in late 2026
文章目录
I get this question every week. "Is one DGX Spark enough?"
Short answer: yes, for a lot more than people think. Here's what I actually run on one box, with real numbers from my own recipes.
Quick note: everything here is on the 128GB Spark. The 64GB model lands on Oct 23, and it won't fit all of these.
Why the DGX Spark works for local AI
The Spark has 128GB of unified memory. CPU and GPU share the same pool.
Compare that to the RTX 5090, the best consumer gaming card, which tops out at 32GB. On one Spark, you can load models the 5090 can't even fit, keep a huge context, and still serve several requests at once.
It's not the fastest box per stream. But it fits things that simply don't fit on consumer GPUs. It also sits quietly on your desk.
What runs well on a single Spark
All numbers are single-stream decode unless I say otherwise. Measured on my Sparks, most of them with sparkDash.
Qwen3.8 Flash Next
Right now, this is the best model you can run on a single DGX Spark. It's fast, it's smart, and it handles pretty much anything you throw at it.
Runs on TensorFold:
-
64.4 tok/s, one stream
-
200.9 tok/s with 8 streams
-
Prefill: 2,630 tok/s on a 32k prompt
-
262k context, up to 8 requests at once
Full details here: https://mia-ai.net/models/Qwen3.8-Flash-Next-Single-DGX-Spark-TensorFold
Qwen3.8 27B
-
TensorFold, MLX 4-bit, DFlash2
-
58.3 tok/s, one stream
-
220.6 tok/s with 8 streams
-
Prefill: 1,633 tok/s on a 32k prompt
-
262k context, 1M with YaRN
Qwen3.6 35B-A3B
-
vLLM, NVFP4, MTP
-
95.1 tok/s, one stream
-
317 tok/s with 8 streams
Upcoming Qwen4 27B
It's not out yet, but it looks close. The biggest hint is this post from @QwenDevs:
What needs 2x DGX Sparks or more
128GB is a lot, but it's not infinite. The big frontier MoEs don't fit on one box.
-
GLM 5.3 Flash: 2 to 4 Sparks. On 2 Sparks with TensorFold it does 60.4 tok/s and 108.8 tok/s with 4 streams.
-
Full GLM 5.3: 3 Sparks, 31.3 tok/s.
-
DeepSeek V4.1 Flash: 2 Sparks with EXL3 2.9-bit (31.6 tok/s), or 3 to 4 Sparks with MXFP4 + FP8 (51.0 tok/s).
If you want these, plan for additional units. Two Sparks linked together is where things get really fun.
Quant tips
-
4-bit is the sweet spot on the Spark. INT4-AutoRound, NVFP4, MLX 4-bit, and EXL3 at 3 to 4 bpw all work well. You keep the quality and get the speed.
-
Smaller can be smarter. For GLM 5.3 Flash, I picked EXL3 over NVFP4. It's smaller, it scores better, and it fits on 2 Sparks much more comfortably.
-
Leave room for context. A model that barely fits leaves no memory for the KV cache, and then you're stuck with short context and one request at a time.
Engine tips
If you want the best performance out of your Sparks right now, use TensorFold. vLLM and SGLang have their strengths, but for pure speed, TensorFold wins.
TensorFold is a local inference engine built by @ashxhart. Most of the fastest numbers in this article run on it. It's moving fast too: the latest update, 1.0.3, added prefix caching, Nvidia Ampere support in beta, and Living Weights, which lets a model learn new facts and keep them in its own weights.
Huge shoutout to Ash 🔥
Get the recipes
Every model here has a free recipe on mia-ai.net/models. Launch scripts, patches and benchmarks are all on GitHub. Clone one and you're running in minutes.
Want to watch your DGX Spark while it works? sparkDash shows GPU, memory, and live tok/s, and it runs the same benchmarks I use.
Go build something with local AI! 🔥
