Ornith-1.5-122B on one DGX Spark

Ornith-1.5-122B is serving on one DGX Spark from a pinned TensorFold image. This lab measured official NVFP4 on Spark nightly vLLM first. The faster EXL3 plus DFlash2 stack is what stayed up.

What's new

Ornith-1.5-122B shipped on 2026-10-20.¹³ The official card is on Hugging Face.¹ Ollama lists the same date.¹⁶

The Hub family is BF16, FP8, NVFP4, GGUF, and MLX. Day-one runtimes named in the announcement include Ollama, Hermes Agent, AtomicChat, Unsloth, RunInfra, OpenCode, Command Code, Inco DFlash2, and SGLang.¹⁴

BF16 usedStorage is 250.19 GB. That does not fit 128 GB of unified memory. The Hub Quickstart line about a single 80 GB GPU is Hub copy. It is not a BF16 VRAM number.

Official NVFP4 does fit.² Hub reports 77.665 GB, which is 72.332 GiB. This lab downloaded 77.7 GB and saw 72.37 GiB on device. We served that checkpoint first.

The standing serve is Mia TensorFold EXL3 plus DFlash2.³⁴⁵ The image is pinned to ghcr.io/miaai-lab/ornith-1.5-122b-tensorfold:zig-d1018b3-f5683f4c5651. Manifest digest is sha256:56c064d22327a6d0ce09d0de3930084b11841918072e9467a2dd9039b3d92b82.

One Spark is enough. A second Spark in the same lab was left on a different model.

The checkpoint config.json architecture is Qwen3_5MoeForConditionalGeneration, model_type qwen3_5_moe: 48 layers, 256 experts, 8 experts per token, native context 262144, one MTP layer, and a vision encoder. That is the same shape Qwen publishes for Qwen3.5-122B-A10B. Ornith describes the 1.5 family as continued training on top of Qwen3.5 and Gemma 4.¹²

Performance

HUB quality, from the official card.¹ This lab did not re-run these scores.

• TerminalBench 2.1: 81.2
• CyberGym: 80.0
• NL2Repo: 51.2
• SWE-bench Pro (no-hack): 76.9
• SWE-bench Verified (non-hack): 84.5
• SWE-bench Multilingual (non-hack): 76.4
• HLE (with tools): 48.0
• HLE (without tools): 37.68
• SciCode: 53.8
• WideSearch: 75.2
• BrowseComp: 78.5
• MCP-Atlas: 74.6
• Toolathlon-Verified: 70.8
• ClawEval: 79.4

Independent GPQA, MMLU, LiveCodeBench, and Arena numbers for this 122B were not published as of 2026-10-21. BenchLM reprints the Hub rows, unranked.¹⁷ GPQA 92.8 is the 397B figure from the August 2026 blog. It is not a 122B score.

LAB, one DGX Spark, 2026-10-21, thinking off, temperature 0.

Standing engine: TensorFold EXL3 plus DFlash2. Image tag zig-d1018b3-f5683f4c5651. EXL3 revision 9c5c913b3de1eef901520d6fedaf450044ea0423, 4.049 bpw, 62 GB on disk, 69.45 GiB on device with the drafter. DFlash2 revision a5ed7396f5f9e5b1abf7ff50b7899a3186961432. CONTEXT 262144. KV fp8. PARALLEL=8.

The engine log showed 8 streams at once, 4.23 GiB each at a 262144-token window, and 36.4 GiB left after a 10 GiB reserve. Load was 112 s on the earlier 4-stream boot. First load is about two minutes. The start script wants about 90 GB MemAvailable before launch.

Thinking is on by default. Callers turn it off with chat_template_kwargs.enable_thinking set to false. Server sampling is 0.6 / 20 / 0.95. Speed runs set temperature to 0.

• One-word check: "What is the capital of France?" answered Paris. Time to first token 0.15 s.
• Prose, 180 completion tokens: 44.1 tok/s decode, TTFT 0.19 s. Engine accepted 106 of 315 drafted tokens.
• Code, fib(n), 200 completion tokens, after the PARALLEL=8 restart: 109 tok/s decode, TTFT 0.21 s. Earlier 4-stream boot, same prompt: 103.5 tok/s.
• Four concurrent short paragraphs, 473 completion tokens in 6.3 s: 75 tok/s aggregate, about 19 to 22 tok/s on each stream.

Official NVFP4 first try, same Spark. Image ghcr.io/spark-arena/dgx-vllm-eugr-nightly, vLLM 0.32.0rc1.dev301 from 2026-10-16. Download 77.7 GB. 72.37 GiB on device. Loaded in 27 s. Flags: quantization modelopt_mixed, Marlin MoE, FlashInfer, FP8 KV, MTP 3 draft tokens, max_model_len 262144, max_num_seqs 4, gpu_memory_utilization 0.85. KV pool 1,428,684 tokens (5.45 times 262144). vLLM warned this path used weight-only FP4 Marlin rather than native FP4.

• Code: 46.7 tok/s, TTFT 0.17 s
• Prose: 33.9 tok/s, TTFT 0.20 s
• Four streams: 61 tok/s aggregate
• MTP accepted 312 of 567 draft tokens

TensorFold stayed because code decode was about 2.2 times faster (109 / 46.7) and prose was faster, while eight full 262144-token windows still fit.

Bitwidth lab, one Spark, 2026-10-21. TensorFold versus official NVFP4. Code 109 vs 46.7 tok/s. Prose 44.1 vs 33.9 tok/s. Four-stream aggregate 75 vs 61 tok/s.

MIA published speeds. These are Mia's numbers. They are not this lab.

Mia's X post on 2026-10-21 listed about 52 tok/s prose single stream, about 103 tok/s code, and about 90 tok/s prose at four streams.⁶ This lab's code figure of 109 tok/s at PARALLEL=8 landed next to Mia's posted ~103 tok/s on code. This lab's four-stream aggregate of 75 tok/s was under Mia's 90 tok/s prose figure.

Mia's TensorFold README table is also Mia's, default PARALLEL=4: prose 52.6 single stream / 90.4 at four streams; code 83.3 single stream / 103.4 at two streams / 111.0 at four streams.⁵ The X post figure of about 103 tok/s code as a single stream is the README's two-stream aggregate of 103.4, not the single-stream 83.3.

INDEPENDENT one-Spark runs used different recipes. They are not this lab.

0xBakeer used TandemLLM 0.13.0 on his 4-bit EXL3 pack. Plain decode about 38 tok/s. MTP plus lookup: prose 85.3, code 114.2, copying 124.8 tok/s.¹⁰

anubhavgupta ran official NVFP4 plus DFlash2 on stock SGLang. One client mixed 39.9 / code 49.1. Four clients mixed 61.4 / code 93.3.¹¹ The same author also posted TensorFold 1.1.1 MLX 4-bit (not Mia's EXL3): 46 / 72 tok/s single stream (mixed / code) and 74 / 98 at four streams.

Local setup

vLLM

The official card's vLLM recipe is a docker run of vllm/vllm-openai:latest. The image entrypoint is vllm serve, so engine args follow the tag. On Spark CDI the device flag is --device nvidia.com/gpu=all.

docker run --device nvidia.com/gpu=all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  --ipc=host \
  vllm/vllm-openai:latest \
  --model ornith-ai/Ornith-1.5-122B \
  --served-model-name Ornith-1.5-122B \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code \
  --tensor-parallel-size 1

That official-card line is for BF16 ornith-ai/Ornith-1.5-122B. usedStorage is 250.19 GB. It does not fit 128 GB. It was not the serve this lab measured.

The measured official-weights serve was ornith-ai/Ornith-1.5-122B-NVFP4 in ghcr.io/spark-arena/dgx-vllm-eugr-nightly. The standing, faster serve is TensorFold ./start.sh with pinned ghcr.io/miaai-lab/ornith-1.5-122b-tensorfold:zig-d1018b3-f5683f4c5651.

The card requires Transformers 5.8.1 or newer, vLLM 0.19.1 or newer, and SGLang 0.5.9 or newer. The NVFP4 card reprints the same BF16 one-liner. There is no NVFP4-specific vLLM command on that card.

The card documents YaRN to about 1M tokens. This lab served the native 262144 window.

LM Studio / llama.cpp

Official GGUF and MLX files exist. This lab did not serve either path.

Official GGUF: Q4_K_M is 75.844 GB, the official GGUF that can sit in 128 GB with KV headroom. Q5_K_M is 88.710 GB (tight). Q6_K is 102.380 GB. Q8_0 is 132.556 GB. BF16 GGUF is 249.363 GB.⁷ The card llama.cpp line is llama-server -hf hf.co/ornith-ai/Ornith-1.5-122B-GGUF --port 8000 -c 262144.

Unsloth Dynamic 3 runs from UD-IQ1_M at 40.4 GB up through Q8_0 at 133 GB and BF16 at 249 GB.⁸ AtomicChat's ladder runs from 132.6 GB (Q8_0) down to 35.0 GB (AD-IQ2_XXS-IQ1_M).⁹ Their 64 GB Mac recommendation is AD-IQ3_XXS-IQ2_S at 47.6 GB, top-1 86.0% versus BF16. That last figure is quant fidelity, not GPQA.

On 128 GB RAM, mid GGUF fits (official Q4_K_M 75.8 GB; Unsloth UD-Q3_K_XL 59.8 GB). AtomicChat Q8_0 at 132.6 GB is tight. Their 128 GiB row cuts context to about 144k. On 32 GB VRAM GPU-only, none of these fit. The smallest published files are AtomicChat 35.0 GB and Unsloth 40.4 GB. Both are over 32 GB.

Official MLX-4bit is 64.02 GiB, the official MLX that can sit in 128 GB. Official MLX BF16-class is about 227.5 GB and does not fit. That path is Apple Silicon. It is not this Spark. This lab served neither.

Point LM Studio at the official GGUF files. No separate LM Studio recipe was measured here.

One DGX Spark

One Spark (GB10, 128 GB unified) is enough. This lab ran the measured serve on one box. A second Spark in the same lab was left on a different model.

What fits:

• Official NVFP4: 77.7 GB download, 72.37 GiB on device. Fits. Tried first. Slower.
• EXL3 4.049 bpw TensorFold: 62 GB disk, 69.45 GiB on device with the drafter. Fits. Standing serve.
• BF16 ornith-ai/Ornith-1.5-122B: does not fit 128 GB.
• Official FP8 at about 133 GB: does not fit with useful KV.
• Official GGUF Q4_K_M at 75.8 GB would fit in RAM. This lab did not serve it.

VRAM floor for the measured GPU paths is about 128 GB unified. Stop the previous model first. The start script wants about 90 GB MemAvailable.

Download the pinned weights and drafter, then pull the image:

hf download Mia-AiLab/Ornith-1.5-122B-EXL3-EAQv1-4-bpw --revision 9c5c913b3de1eef901520d6fedaf450044ea0423
hf download incoai/Ornith-1.5-122B-DFlash2 --revision a5ed7396f5f9e5b1abf7ff50b7899a3186961432
docker pull ghcr.io/miaai-lab/ornith-1.5-122b-tensorfold:zig-d1018b3-f5683f4c5651

Working measured command. On a first run, leave PREPARE=auto so the script pulls the image and weights. After they are cached:

git clone https://github.com/MiaAI-Lab/Ornith-1.5-122B-TensorFold.git
cd Ornith-1.5-122B-TensorFold

PREPARE=0 \
PORT=8000 \
CONTEXT=262144 \
PARALLEL=8 \
KV_DTYPE=fp8 \
DRAFTS=1 \
DRAFTER=dflash2 \
CONTAINER_NAME=sparkrun-ornith-mia \
./start.sh

Pin the image tag. The measured serve is that tag, not latest. The tag this lab ran is ghcr.io/miaai-lab/ornith-1.5-122b-tensorfold:zig-d1018b3-f5683f4c5651. Weights revision 9c5c913b3de1eef901520d6fedaf450044ea0423. Drafter revision a5ed7396f5f9e5b1abf7ff50b7899a3186961432. GHCR is public. Manifest digest is sha256:56c064d22327a6d0ce09d0de3930084b11841918072e9467a2dd9039b3d92b82.

start.sh launches docker run with --gpus all. On a Spark that only exposes CDI, a raw docker run would use --device nvidia.com/gpu=all. The measured serve is ./start.sh, not a handwritten docker line.

Smoke on localhost:

curl -s http://127.0.0.1:8000/v1/models

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Ornith-1.5-122B",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "max_tokens": 32,
    "temperature": 0,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

The one-word check answered Paris. Time to first token 0.15 s.

Two DGX Sparks

This lab did not use a second Spark. There is no measured tensor-parallel recipe here.

Mia's TensorFold README says tensor parallel on two Sparks (TP2) is untested.

A second Spark would copy the one-box TensorFold command. It would not split the model. The official card --tensor-parallel-size flag is for BF16 vLLM or SGLang across more GPUs. It is not a measured two-Spark serve of this checkpoint.

Inco DFlash2 eval numbers on two NVIDIA GB300 at TP=2 are a different box. They are not Spark.¹⁵

Desktop GPU (5090 and below)

62 GB EXL3 and 72 GiB NVFP4 do not fit 32 GB.

• TensorFold EXL3: 62 GB disk, 69.45 GiB on device with the drafter.
• Official NVFP4: 72.37 GiB on device.
• RTX 5090 is 32 GB. Weights alone exceed the card.

GGUF on CPU plus RAM is a different path. GPU-only on 32 GB: even the smallest published GGUF (AtomicChat 35.0 GB, Unsloth 40.4 GB) does not fit VRAM. A 5090 might offload layers to system RAM via llama.cpp -ngl. This lab did not measure that. There is no 5090 tok/s figure here.

Hardware fit for Ornith-1.5-122B. One 128 GB Spark: FIT (EXL3 62 GB disk / 69.45 GiB with drafter; NVFP4 72.37 GiB). BF16 250.19 GB: no fit. Two Sparks: not used. Desktop GPU 5090 and below (32 GB): no fit.