Dual DGX Spark & the Unified AI Gateway Overhaul

Back in March I wrote about my AI stack built around OWUI as a unified gateway. That post described the destination: applications point at model aliases, not backends, and the gateway routes to whatever is serving the model.

What it did not cover was the why behind the dual DGX Spark setup, or everything that changed on the hardware side since then. This post fills in those gaps: why I bought the second Spark, whether the bet paid off, how the rest of the GPUs got reshuffled, and how the alias strategy has matured into the thing that makes the whole stack easy to live with.

The Why: Adding the Second DGX Spark (Feb 2026)

In February 2026 I added a second DGX Spark to the stack. It is an identical ASUS GX10.

At the time, I felt that models which fit into 128GB lacked a lot of “smarts”. Knowledge depth and reasoning were noticeably weaker than the big frontier models, and it was not really the models’ fault. We were still in the early phases of models being optimized for that memory range. Quantization aware quantized models where still rare and multi-token prediction was yet to be super popular.

Did the Bet Pay Off?

Yes, decisively.

Once the second Spark was in, the jump in LLM quality and knowledge was massive. At the time of writing, models that fit into 2 sparks where getting released frequently minimax-2.7, then deepseek-v4-flash, and later the RL-tuned deepseek-v4-flash-0731. Then came nvidia/GLM-5.3-Flash-FP4-Local, which at the time of writing it is the model being used, which runs on dual Sparks at roughly 99% the quality of its FP8 sibling. That model is what anchors the gateway today, and it is a genuinely different class of model than anything that fit on a single Spark back in February.

As with my earlier posts, a huge thank you to the eugr/spark-vllm-docker project. The recipe I run is glm-5.3-flash.yaml, and it has been rock solid.

There is also a timing angle to this story, and it is pure luck. I bought both Sparks before DGX Spark prices roughly doubled. Tight inventory and demand pressure on the chip and memory industry have pushed prices up since then, so I happened to get in ahead of the jump.

The Plot Twist (Oct 2026)

Here is the honest retrospective part. Since February, model architecture has changed significantly:

• Better NVFP4/FP4 quantization: quality per GB of memory went way up, so models no longer need to be traded down as aggressively to fit.
• Better MTP options: things like dflash and dspark improved throughput without needing exotic setups.

The result is that a single Spark is now genuinely capable in a way it was not in early 2026. If you are a solo Spark user reading this in October 2026, do not rush out and buy a second unit. My current recommendation for a single Spark is the qwen3.8-flash-next-nvfp4-solo.yaml recipe. It is excellent.

Would I still buy the second Spark today knowing what I know now? Ignoring the price increase, for my use case, yes, because I want the big-model tier resident for agentic work. But the gap between one Spark and two is much smaller than it was nine months ago, and that is worth knowing before you spend the money.

The GPU Reshuffle

While the second Spark was the headline change, the rest of the stack got reorganized around a simple principle: give each workload the right GPU, and keep hot paths off of pools that swap models.

Dual DGX Sparks: “big”

The Sparks run vLLM and host the largest and smartest models, currently GLM-5.3-Flash-FP4. Everything in this tier is always resident. No swapping, no cold starts, just consistent capacity for the heavy agentic workloads.

Unraid: Ollama on-demand plus a dedicated voice GPU

The Unraid server now carries four GPUs with two distinct jobs:

• 3× RTX 5060 Ti 16GB run Ollama for on-demand models. These serve the small and fast tier. Models load when needed and unload after inactivity, so the pool stays available for whatever experiment I am running that week.
• 1× RTX 3060 Ti is dedicated to live voice in and out. Voice is latency sensitive in a way text is not, and I did not want voice conversations fighting with Ollama model loads and swaps for GPU time. Giving voice its own GPU keeps it consistent no matter what the 5060 Ti pool is doing.

Voice runs on speaches (ghcr.io/speaches-ai/speaches:0.9.0-rc.3-cuda), which serves both sides of the pipeline:

• STT: istupakov/parakeet-tdt-0.6b-v3-onnx
• TTS: speaches-ai/Kokoro-82M-v1.0-ONNX-fp16

One practical note on this box: all the GPUs are power limited, and the gpu core and vram memory speeds are underclocked to reduce heat. None of the GPUs are blower models, and they are tightly packed together in the case, so thermals need all the help they can get. Despite the aggressive power limits, I have noticed minimal impact on token speed, which tells you how much headroom these cards leave on the table at stock settings.

Dev Box: image generation

The dev box with an RTX 5080 runs image generation through Unsloth Studio, exposing an OpenAI-compatible images endpoint. The current model is unsloth/Qwen-Image-2.1-Turbo-GGUF. Image generation is bursty rather than constant, so it lives on the dev box instead of occupying space in the Unraid pool.

The Playground

The Dell 5810 workstation (10.0.0.8) has a 3060 Ti, a 3070 Ti, and 256GB of system RAM. This is the experimental sandbox: new inference engines, odd model formats, half-broken ideas. When something there proves itself, it graduates to the real stack. It also exposes an OpenAI-compatible endpoint that OWUI can route to, which makes A/B testing new models against production traffic trivial.

One Gateway, Many Clients: The Alias Strategy

All of this hardware sits behind one OWUI instance, and that is where the architecture actually earns its keep.

The New Traffic Flow

Traefik terminates TLS for ai.rushdynamics.com and hands everything to OWUI. From there, requests split down two paths: the LLM APIs (vLLM on the Sparks for big, Ollama on Unraid for small, plus the Dell 5810 playground) and the OpenAI-compatible multimodal APIs (speaches for voice, Unsloth Studio for images). Clients only ever see the aliases.

Who uses it:

• The family: my wife and kids each have accounts. The kids get a model alias named kids that points at a model with a custom system prompt focused on safety and education. Same gateway, same hardware, but a curated experience per audience.
• Me: OpenCode, Hermes and Hermes desktop, ChatGPT/Codex desktop, and Claude Code, all pointed at the same endpoint.
• Friends: a few friends also use my stack to experiment with local AI before they take the plunge and buy their own hardware. One of them already has a Spark of his own, and sometimes he needs the smarter models that only the dual Sparks can host. Or sometimes he just likes to compare the models he is running against the ones I am running.

The pattern is the continuation of the workspace model idea from the March post, pushed to its logical conclusion. Instead of client configs naming real models, everything uses two aliases:

• big: whatever the dual DGX Sparks are hosting, since they can host the largest and smartest models. The Hermes agent alias points here too.
• small: small and fast models. Right now I am experimenting with nemotron-3.5-bolt, qwen3.6:35b, and nemotron-3.5-lightning.

The payoff is exactly what you would hope for: when a new model drops, I swap one alias in OWUI and every client in the house picks it up. No config changes anywhere.

Config Examples

The following snippets show the alias pattern with real values replaced by big and small, which is also how I think about them mentally. Note that both clients point at the OWUI gateway, not directly at any backend.

OpenCode

cat ~/.config/opencode/opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "model": "rush-ai/big",
  "provider": {
    "rush-ai": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Rush AI",
      "options": {
        "baseURL": "https://llm.mywebsite.com/api/v1",
        "apiKey": "sk-MY-SECRET-TOKEN"
      },
      "models": {
        "big": {
          "name": "big",
          "tool_call": true,
          "reasoning": true,
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          },
          "limit": {
            "context": 500000,
            "output": 131072
          },
          "variants": {
            "max": { "reasoningEffort": "max" },
            "low": { "reasoningEffort": "low" },
            "high": { "reasoningEffort": "high" }
          }
        },
        "small": {
          "name": "small",
          "tool_call": true,
          "limit": {
            "context": 262144,
            "output": 131072
          },
          "variants": {
            "low": { "reasoningEffort": "low" },
            "medium": { "reasoningEffort": "medium" },
            "xhigh": { "reasoningEffort": "xhigh" }
          }
        }
      }
    }
  },
  "agent": {
    "build": {
      "options": {
        "temperature": 0.7
      }
    },
    "plan": {
      "options": {
        "temperature": 0.8
      }
    }
  }
}

Claude Code

cat ~/.claude/settings.json
{
  "autoCompactEnabled": true,
  "env": {
    "ANTHROPIC_BASE_URL": "https://llm.mywebsite.com/api",
    "ANTHROPIC_AUTH_TOKEN": "sk-MY-SECRET-TOKEN",
    "ANTHROPIC_MODEL": "big",
    "ANTHROPIC_SMALL_FAST_MODEL": "small",
    "CLAUDE_CODE_ENABLE_TELEMETRY": "0",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "CLAUDE_CODE_ATTRIBUTION_HEADER": "0",
    "CLAUDE_AUTOCOMPACT_PCT_OVERRIDE": "80",
    "CLAUDE_CODE_MAX_CONTEXT_TOKENS": "262144",
    "CLAUDE_CODE_DISABLE_1M_CONTEXT": "1"
  },
  "attribution": {
    "commit": "",
    "pr": ""
  }
}

OWUI Configuration

Here is the current OWUI container configuration

# Ollama
'ENABLE_OLLAMA_API'='true'
'OLLAMA_BASE_URL'='http://10.0.0.10:11434'
# Vllm
'ENABLE_OPENAI_API'='true'
'OPENAI_API_BASE_URL'='http://10.0.0.8:8000/v1;http://10.0.0.11:8000/v1'
# Web search
'ENABLE_WEB_SEARCH'='true'
'WEB_SEARCH_ENGINE'='tavily'
'TAVILY_API_KEY'='<redacted>'
'TAVILY_EXTRACT_DEPTH'='advanced'
'WEB_SEARCH_RESULT_COUNT'='4'
# Audio
'AUDIO_TTS_ENGINE'='openai'
'AUDIO_TTS_OPENAI_API_BASE_URL'='http://10.0.0.10:8000/v1'
'AUDIO_TTS_OPENAI_API_KEY'='1234'
'AUDIO_TTS_MODEL'='speaches-ai/Kokoro-82M-v1.0-ONNX-fp16'
'AUDIO_TTS_VOICE'='am_puck'
'AUDIO_STT_ENGINE'='openai'
'AUDIO_STT_OPENAI_API_BASE_URL'='http://10.0.0.10:8000/v1'
'AUDIO_STT_OPENAI_API_KEY'='1234'
'AUDIO_STT_MODEL'='istupakov/parakeet-tdt-0.6b-v3-onnx'
# Image
'ENABLE_IMAGE_GENERATION'='true'
'IMAGE_GENERATION_MODEL'='unsloth/Qwen-Image-2.1-Turbo-GGUF'
'IMAGES_OPENAI_API_KEY'='<redacted>'
'IMAGES_OPENAI_API_BASE_URL'='http://10.0.0.25:8888/v1'
'IMAGE_GENERATION_ENGINE'='openai'
'IMAGE_SIZE'='512x512'
# General
'BYPASS_MODEL_ACCESS_CONTROL'='true'
'USER_PERMISSIONS_CHAT_CONTROLS'='false'
'USER_PERMISSIONS_WORKSPACE_MODELS_ACCESS'='true'
'CUSTOM_NAME'='Rush.Ai'
'WEBUI_NAME'='Rush.Ai'
'ENABLE_SIGNUP'='false'
'ENABLE_API_KEYS'='true'
'ENABLE_PERSISTENT_CONFIG'='false'
'DEFAULT_GROUP_ID'='everyone'

Routing

The interesting lines are the backend URLs:

• OPENAI_API_BASE_URLS lists the OpenAI-compatible backends: the Dell 5810 playground (10.0.0.8) and the DGX Spark (10.0.0.11). This is how the big alias can live on the Sparks while experimental models on other hardware are reachable from the same gateway.
• OLLAMA_BASE_URL points at Ollama on Unraid, which backs the small tier and the on-demand model pool.

Model fallbacks are configured in the OWUI admin settings, so if a backend is down or a model is unloaded, requests fail over instead of erroring out.

Auth and Access Control

• WEBUI_AUTH=‘true’ with ENABLE_SIGNUP=‘false’: accounts exist for exactly the people I invited, and nobody else can join.
• ENABLE_API_KEYS=‘true’: every client and person gets their own token, which is what makes the alias-per-audience setup enforceable.
• DEFAULT_GROUP_ID=‘everyone’ and USER_PERMISSIONS_WORKSPACE_MODELS_ACCESS=‘true’: workspace aliases are visible to all accounts.
• BYPASS_MODEL_ACCESS_CONTROL=‘true’: an honest tradeoff. This is a family gateway on a trusted LAN, so I opted for simplicity over per-model ACLs. If you are exposing your gateway beyond your network, think carefully before copying this.
• ENABLE_ADMIN_CHAT_ACCESS and ENABLE_ADMIN_ANALYTICS: as the admin I can see usage across accounts, which is useful when the kids report that “the AI is being weird”.

Voice

Both audio engines point at speaches on the dedicated 3060 Ti (10.0.0.10:8000), with Parakeet handling speech-to-text and Kokoro handling text-to-speech. Because the endpoint is OpenAI-compatible, OWUI treats it like any other OpenAI service, which keeps the voice config to a handful of environment variables.

Image Generation

The image pipeline is the same story: an OpenAI-compatible endpoint (openai engine) pointed at Unsloth Studio on the dev box.

Search

Web search uses Tavily with advanced extract depth and four results per query. Advanced depth costs a bit more per search but returns noticeably cleaner content for the model to work with.

Summary

Six months after the unified gateway post, the architecture held up, and this round of changes was almost entirely additive:

• The dual Spark bet paid off: deepseek-v4-flash, deepseek-v4-flash-0731, and GLM-5.3-Flash-FP4-Local are a different tier than what a single 128GB box could host in early 2026.
• The landscape shifted: better NVFP4/FP4 quantization and MTP options like dflash and dspark mean a solo Spark is now a legitimate setup, with the Qwen3.8-Flash-Next solo recipe as my recommendation for single-Spark users.
• Every GPU has one job: Sparks for big always-resident inference, 5060 Ti pool for on-demand small models, dedicated 3060 Ti for voice, 5080 for images, and the Dell 5810 for experiments.
• Aliases make it all survivable: big, small, and kids absorb every model swap, so clients and family members never notice the infrastructure churning underneath them.

If you want the full backstory, the trilogy so far is the arrival post, the initial benchmarks, and the unified gateway architecture.