SGLang vs vLLM on DGX Spark: Which to Use?

For SGLang vs vLLM on DGX Spark, the short answer is this: pick vLLM if you want the widest set of models NVIDIA has validated on the Spark, official recipes for linking two or four Sparks, and the best results when many requests arrive at once. Pick SGLang if one person or one agent is talking to the model most of the time, your prompts repeat a long shared prefix, or you want speculative decoding to push single stream speed past what the memory bandwidth normally allows. Both are free, both expose an OpenAI compatible API, and both run from a CUDA 13 container on the Spark, so trying the other one later costs you an afternoon, not a rebuild.

The rest of this guide explains why the Spark changes the usual comparison, what each engine needs to start, and how to choose by workload. If you want the general SGLang vs vLLM comparison on ordinary GPU servers, read our SGLang vs vLLM vs llama.cpp guide instead. This page is only about NVIDIA's GB10 desktop.

SGLang vs vLLM on DGX Spark at a glance

The container names, flags and model lists come from NVIDIA's vLLM playbook and SGLang playbook for DGX Spark. Model counts change as NVIDIA adds rows, so check both lists before you download anything large.

Why the DGX Spark changes the comparison

On a rack of H100s, the SGLang vs vLLM argument is mostly about scheduling and kernels. On a Spark, the hardware sets a different limit. According to NVIDIA's DGX Spark product page, the GB10 Grace Blackwell chip pairs a 20 core Arm CPU (10 Cortex-X925 and 10 Cortex-A725 cores) with a Blackwell GPU rated at up to 1 petaFLOP of sparse FP4 compute. It shares 128 GB of LPDDR5x memory between CPU and GPU (a 64 GB version is sold only through OEM partners), and that memory has 273 GB/s of bandwidth.

That bandwidth figure is what you feel. Every generated token has to read the active weights from memory, so a large dense model generates slowly no matter which server you use. The SGLang team's own DGX Spark review at LMSYS said exactly this: memory bandwidth was the key bottleneck in its tests, and Llama 3.1 70B in FP8 decoded at 2.7 tokens per second at batch 1. The same review showed the other side. Llama 3.1 8B in FP8 went from 20.5 tokens per second for one request to 368 tokens per second in total at batch 32, because batching reuses each weight read across many requests.

Three practical consequences follow:

1. Model choice matters more than engine choice. Mixture of experts models with few active parameters (gpt-oss-120b, Nemotron 3 Super 120B A12B, Qwen3.6 35B A3B) and NVFP4 weights do far more for speed than switching servers. Our VRAM guide explains how to size weights and cache.
2. Batching is where vLLM earns its place. If several people or processes share the Spark, continuous batching turns the bandwidth limit into higher total throughput.
3. Speculative decoding is where SGLang earns its place. A draft model proposes several tokens and the main model checks them in one pass, which gets more work out of each memory read. LMSYS reported up to a 2x end to end speedup with EAGLE3 on the Spark. Our speculative decoding vs MTP guide explains how this works. vLLM also supports speculative decoding, so treat this as a difference in how much tuning each project has published for the Spark, not a feature only one has.

Setting up vLLM on DGX Spark

NVIDIA's vLLM playbook uses its own NGC build of vLLM, compiled for the Spark's ARM64 CPU and CUDA 13. You pick the newest tag from the NGC catalogue, pull it and start the server:

export HF_TOKEN="your_huggingface_token"
export LATEST_VLLM_VERSION=26.05.post1-py3
docker pull nvcr.io/nvidia/vllm:${LATEST_VLLM_VERSION}
docker run -d --gpus all -p 8000:8000 --name vllm-server \
  nvcr.io/nvidia/vllm:${LATEST_VLLM_VERSION} \
  vllm serve openai/gpt-oss-20b

A few details from the playbook are easy to miss:

• Gemma 4 uses a separate image. NVIDIA points Gemma 4 models at vllm/vllm-openai:gemma4-cu130 rather than the NGC build.
• Some models need extra flags. Phi-4 multimodal needs --trust-remote-code, and Nemotron 3 Nano Omni has its own launch notes on its model card.
• Out of memory usually means too much context. The troubleshooting table says to lower --max-model-len and --max-num-seqs before anything else.
• Clusters are covered. The same playbook has step by step sections for two Sparks over a direct QSFP cable and for several Sparks through a switch, both using Ray and tensor parallelism.

NVIDIA's vLLM page on build.nvidia.com also offers an easier path through NVIDIA Sync on your laptop, with suggested models for both the 64 GB and 128 GB Spark.

Setting up SGLang on DGX Spark

The SGLang playbook uses the project's own CUDA 13 image. You start the container, then launch the server inside it:

export SGLANG_IMAGE="lmsysorg/sglang:latest-cu130"
docker pull "$SGLANG_IMAGE"
docker run --gpus all -it --rm -p 30000:30000 -v /tmp:/tmp "$SGLANG_IMAGE" bash

# inside the container
python3 -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V2-Lite \
  --host 0.0.0.0 --port 30000 --trust-remote-code \
  --tp 1 --attention-backend flashinfer --mem-fraction-static 0.75

For NVIDIA's NVFP4 checkpoints, such as nvidia/Llama-3.3-70B-Instruct-FP4 or nvidia/Qwen3-32B-FP4, add --quantization modelopt_fp4. NVIDIA's SGLang page on build.nvidia.com adds a multi turn benchmark script that shows RadixAttention reusing cached prefixes, plus a structured JSON output test using xGrammar.

If you want speculative decoding, the LMSYS review includes a full EAGLE3 launch command for Llama 3.1 8B with a published draft model. Expect to tune the number of draft steps and tokens for your own model, and to lower the memory fraction so the draft model fits.

One memory quirk that hits both

Both playbooks carry the same warning. DGX Spark uses a unified memory architecture, and because many applications are still adapting to it, you can hit memory errors even when the model should fit. NVIDIA's fix is to flush the Linux page cache before starting the server:

sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

This matters most right after a large download, when the operating system is holding gigabytes of file cache that the GPU allocator does not count as free. It also explains the memory flags. On a normal GPU, giving vLLM 90% of VRAM is routine. On a Spark, the CPU, Docker and your desktop share the same pool, which is why NVIDIA's examples stay at 0.75 to 0.8. If you want to stretch context further, our KV cache quantization guide shows how to shrink the cache instead of raising the limit.

Which is faster on DGX Spark?

There is no official head to head benchmark from NVIDIA, so be careful with any single number. What public tests agree on is the pattern, not the exact gap.

One independent test by Sagar Pandita ran the same NVFP4 checkpoint of Qwen3.8-Flash-Next under both engines on one Spark, with prefix caching turned off. SGLang decoded roughly 18 to 22% faster for a single request across the context sweep, and prompt processing was within a few percent at the largest sizes. SGLang also led total throughput at one, two and four parallel streams, but vLLM won at eight streams, 162.7 to 151.5 tokens per second. The author notes it was one Spark, one run per context size and pinned development images.

That matches what the engines are designed for. SGLang tends to win when few requests are in flight and when prompts share a prefix that RadixAttention can reuse. vLLM tends to pull ahead as concurrency rises. Your result will depend on the model, the quantization and the container version, so the honest advice is to test your own workload with both. Each server takes minutes to start once the weights are on disk.

How to choose by workload

That last row is worth taking seriously. NVIDIA also publishes DGX Spark playbooks for llama.cpp, Ollama and LM Studio. If you are the only user and you prefer GGUF files, those tools are simpler to run, and our Ollama vs llama.cpp vs LM Studio comparison covers the tradeoffs. Move to vLLM or SGLang when you need NVFP4 checkpoints, real batching, an agent framework that expects a production style server, or speculative decoding with tuned draft models.

Good first models for a 128 GB Spark

NVIDIA's product page says one 128 GB Spark can run inference on models up to 200 billion parameters, and a 64 GB unit up to 100 billion. Within that, these picks appear on both validated lists and are a sensible way to compare the two servers on equal terms:

• gpt-oss 20b and 120b (MXFP4). Small active parameter counts make them responsive on the Spark. Our gpt-oss local guide covers the reasoning settings.
• Qwen3 8B, 14B and 32B (FP8 or NVFP4). Good for checking how much NVFP4 speeds things up against FP8 on the same model.
• Llama 3.3 70B (NVFP4). A dense 70B model, useful to see the bandwidth limit for yourself. Expect it to be slow for one user on either engine.

If you lean towards vLLM, also try Gemma 4 26B A4B or Nemotron 3 Super 120B A12B, both on NVIDIA's vLLM list. To make your own NVFP4 checkpoint for a model NVIDIA has not published, see NVIDIA's NVFP4 quantization playbook and our guide on how to quantize LLM models.

FAQ

Does vLLM work on DGX Spark?

Yes. NVIDIA maintains a DGX Spark playbook for vLLM that uses an NGC container built for the Spark's ARM64 CPU and CUDA 13, with a list of validated models and recipes for one, two or several linked Sparks.

Does SGLang work on DGX Spark?

Yes. NVIDIA's SGLang playbook runs the lmsysorg/sglang:latest-cu130 image on a single Spark and lists validated models, including gpt-oss, Llama 3.3 70B in NVFP4 and Qwen3. The SGLang team published its own Spark review and launch commands as well.

Is SGLang faster than vLLM on DGX Spark?

Often for a single request, in public tests. One independent test found SGLang about 18 to 22% faster at single stream decode on the same NVFP4 model, while vLLM won total throughput at eight parallel streams. Test your own model, because versions and quantization change the result.

Can I run SGLang or vLLM across two DGX Sparks?

NVIDIA's vLLM playbook documents two Sparks over a direct QSFP cable and larger setups through a switch, using Ray and tensor parallelism. NVIDIA's SGLang playbook covers a single Spark, so a multi Spark SGLang setup means following SGLang's own distributed docs and testing carefully.

Why does my server run out of memory on a DGX Spark when the model should fit?

The Spark's CPU and GPU share one memory pool, and the Linux file cache can hold memory the server expects to be free. NVIDIA's playbooks suggest flushing the cache with drop_caches, then lowering context length or the number of parallel sequences if the error persists.

Should I use Ollama instead of vLLM or SGLang on a DGX Spark?

For one person chatting with GGUF models, Ollama or llama.cpp is simpler. Choose vLLM or SGLang when you need NVFP4 checkpoints, batching for several users, or speculative decoding. See our SGLang vs vLLM vs llama.cpp guide for the wider comparison.