Start with one AI Cube and expand when required? The AI Cube is designed as a productive single-system starting point. For more users or larger models, we connect and configure a second AI Cube for the measured workload. Explore the AI Cube and expansion paths
GB10 systems can connect directly through ConnectX-7. This enables two distinct architectures: multiple model endpoints for more concurrent requests, or one model distributed across nodes. Calling both a “cluster” hides different goals and bottlenecks.
Hardware foundation
Each GB10-based AI Cube has 128 GB of unified memory, 10 GbE, and two QSFP connectors attached to the ConnectX-7 NIC. NVIDIA documents up to 200 Gbit/s per QSFP port and recommends compatible cables. These interfaces operate as Ethernet; they do not create shared memory between systems.
Two devices can use a direct cable. For DGX Spark, the NVIDIA Sync Cluster Assistant also supports three directly connected systems and up to four through a switch. The AI Cube is based on the ASUS Ascent GX10, and ASUS states at most two devices in a cluster. Three- and four-node topologies therefore apply to DGX Spark and other OEM systems such as Acer Veriton GN100; for AI Cubes they are only an option after individual review. Larger or different topologies need a custom network design. Cabling, firmware, MTU, addressing, and topology must be documented.
Option A: more independent requests
Each AI Cube loads a model and serves its own requests. A load balancer sends new sessions or API calls to available endpoints. This increases aggregate throughput, allows different models per node, and can isolate failures.
Open WebUI, database, knowledge store, and session state still need a defined location. Duplicated inference alone is not a highly available platform.
Option B: one model across systems
If a model and KV cache do not fit usefully on one node, a supported runtime can distribute weights and computation. Nodes exchange intermediate results over ConnectX-7. Latency and throughput then depend on model architecture, parallelisation, runtime, and network.
Two devices do not become one computer with 256 GB of freely addressable memory. Each provides 128 GB coordinated by distributed software, so the exact model artefact must be tested.
Direct two-node connection
A production setup includes compatible operating-system, driver, and firmware levels; a validated QSFP cable; explicit IP addresses; connectivity and throughput tests; runtime configuration; workload measurement; and monitoring for nodes, network, model, and queues.
NVIDIA notes that one physical port can appear as several Linux interfaces because of the internal topology. Use the documented port mapping rather than guessing from interface names.
What NVIDIA Sync handles - and what it does not
The NVIDIA Sync Cluster Assistant discovers devices, creates a supported ConnectX-7 topology, applies network settings, validates links, and configures SSH between nodes. This establishes the communication layer.
It does not configure model distribution, NCCL or vLLM, container orchestration, load balancing, or high availability. NVIDIA has announced the NVIDIA Sync Model Launcher for the end of October 2026; it is intended to download and launch selected models on a single system or a cluster. Until it is available and tested for the specific workload, model deployment remains a separate task, and even then it replaces neither load balancing nor high availability. Those layers still need to be designed and tested for the model and operating objective. A successful network test is not yet a working inference cluster.
Decision matrix
| Goal | First approach |
|---|---|
| more concurrent users | independent model endpoints and load balancing |
| larger single model | distributed inference with exact model testing |
| several specialist models | separate models by node and route deliberately |
| committed availability | make the complete platform redundant |
| three nodes | directly cabled with NVIDIA Sync for DGX Spark; for AI Cubes (ASUS GX10, max. two per ASUS) only after individual review |
| four nodes | switch topology for DGX Spark and supported OEM systems; for AI Cubes only after individual review |
| more than four nodes | custom network and orchestration design required |
WZ-IT expansion path
With the AI Cube, one local workload is first configured and measured. As use grows, we add a second AI Cube for throughput or distributed models. When rack mounting, redundant power, several discrete GPUs, or stricter availability are needed, AI Cube Custom or a GPU server is the better path.
Before scaling out, define both the usage and load profile and the specific model artefact for 128 GB. Otherwise, hardware is connected before the actual capacity problem has been identified.
