Dual DGX Spark Cluster Blueprint

> **Your Outcome:** A production-ready dual-node DGX Spark cluster serving LLMs with tensor parallelism across two GB10 boxes, fronted by LiteLLM, monitored by Prometheus/Grafana, and deployed entirely via Ansible — powered by 200Gbps RoCE interconnect. ## The Problem You have one or two DGX Spark (GB10) systems on your desk and you know they're capable of more than what a single node delivers. But stitching two of them into a coherent inference cluster means navigating NVIDIA's networking stack, tuning NCCL for NVLink-over-RoCE, wrangling Ansible roles for vLLM and LiteLLM, and building monitoring from scratch — weeks of trial-and-error that nobody documents end-to-end. The official DGX Spark documentation covers single-node usage. The community forums have half-answers about RoCE. Nowhere will you find a complete, tested blueprint that goes from stock GB10s to a production inference cluster with tensor parallelism, a unified API proxy, and full observability. ## What This Guide Does For You This blueprint delivers the complete Ansible-based deployment that took months to develop and battle-test. Clone the repo, edit your inventory, and run one playbook. Two hours later you have a dual-node vLLM cluster with TP2, LiteLLM routing, 200Gbps RoCE fabric, Prometheus metrics, and Grafana dashboards — all configured and talking to each other. No guesswork. No forum-scavenging. No figuring out which NCCL environment variables actually matter for GB10. ## What You'll Be Able To Do - **Deploy a dual DGX Spark cluster from scratch** — stock Ubuntu, one Ansible run, zero manual SSH gymnastics - **Serve models with tensor parallelism across two nodes** — vLLM configured for TP2 over 200Gbps RoCE, splitting larger models across both GB10s - **Route inference through LiteLLM** — unified OpenAI-compatible endpoint with load balancing, fallbacks, rate limiting, and cost tracking - **Build a 200Gbps RoCE fabric** — NCCL environment tuning, fabric manager config, PFC/ECN setup, and bandwidth validation - **Monitor everything that matters** — Prometheus node + GPU + vLLM metrics, Grafana dashboards for cluster health, throughput, latency, and memory pressure - **Scale the pattern** — the same Ansible roles adapt to 4, 8, or more nodes with minimal changes - **Validate the cluster** — included smoke tests confirm NCCL communication, RoCE bandwidth, and end-to-end inference before you declare victory ## Who Will Benefit Most Infrastructure engineers, ML platform teams, and AI researchers who own DGX Spark hardware and need to extract maximum inference performance from their investment. You know your way around Linux and YAML — this blueprint gives you the NVIDIA-specific depth without the painful experimentation. ## What Success Looks Like Your two GB10s are no longer standalone boxes. They're a unified inference cluster serving production-grade LLMs through a single API endpoint, monitored and measured, deployable in under two hours, and backed by an Ansible codebase you can version-control, audit, and extend. When someone asks "what's the throughput on the Spark cluster?", you pull up a Grafana dashboard instead of guessing. ## Format & Delivery - Complete Ansible playbook with roles for vLLM, LiteLLM, Prometheus, Grafana, and RoCE networking - Ansible inventory templates and group variables for dual-node and multi-node topologies - NCCL environment tuning guide specific to GB10's Grace Hopper architecture - RoCE fabric validation playbook with bandwidth benchmarks - Grafana dashboards exported as JSON (import and go) - Immediate digital download (.zip) — unzip, edit inventory, run playbook