Tag: inference

Run 3 specialised LLMs on a single DGX Spark in under 2 minutes with 100+ tok/s throughput. Production orchestration patterns revealed.

DeepSeek V4 ships two open-weight MoE models — a 1.6T Pro and a 284B Flash — with novel sparse attention, FP4 quantisation, 1M token context, and validated Huawei Ascend NPU support. Here's what actually changed.

A technical comparison of vLLM and SGLang, the two leading open-source LLM inference engines, covering architecture, performance, and when to pick each one.

A practical guide to deploying production-ready LLM inference using vLLM on NVIDIA DGX Spark hardware, covering configuration, troubleshooting, and performance optimization.