Configure an AI Cube with GPT-OSS 120B - fully configured and hardened GB10 appliance, Open WebUI, an agreed local model, initial setup and support. Schedule an initial consultation
OpenAI released GPT-OSS 120B as an open-weight reasoning model on 5 August 2025. Its native MXFP4 quantisation allows OpenAI to position the model for a single GPU with 80 GB of memory. This also makes it relevant for compact systems with 128 GB unified memory.
The key point is that loading a model into memory is not the same as operating a suitable multi-user system. Context length, concurrent requests, runtime and required response time must fit together.
Table of contents
- What defines GPT-OSS 120B
- Does GPT-OSS 120B fit the AI Cube?
- What is configured before delivery
- Open WebUI, Ollama or vLLM
- RAG and company knowledge
- Limits of the GB10 class
- Decision
What defines GPT-OSS 120B
GPT-OSS 120B is a mixture-of-experts model. OpenAI states that 5.1 billion parameters are active per token, reducing compute compared with a dense model of the same total size.
| Property | GPT-OSS 120B |
|---|---|
| Model type | Text reasoning model with mixture of experts |
| Active parameters per token | 5.1 billion |
| Weight format | MXFP4 |
| Memory class stated by OpenAI | One 80 GB GPU |
| Licence | Apache 2.0 |
| Interfaces | Designed in part for Responses API-compatible workflows |
Open weight means the weights can be run and adapted locally. It does not mean that the full training dataset and training process are public.
Does GPT-OSS 120B fit the AI Cube?
The WZ-IT AI Cube uses NVIDIA GB10 with 128 GB unified memory shared between CPU and GPU. The MXFP4 weights of GPT-OSS 120B therefore fit the available memory class in principle.
For a useful deployment, we also assess:
- required context length,
- expected concurrent requests,
- typical answer length,
- RAG context and tool schemas,
- support in the selected runtime,
- required response time.
Longer context and more concurrent requests consume additional memory beyond the weights. We therefore do not promise a fixed user count based only on the 128 GB specification.
What is configured before delivery
The local model is not selected on the fly at the customer site. Before ordering, we discuss the workload and agree whether GPT-OSS 120B or a smaller model is more appropriate.
The AI Cube delivery includes:
- complete initial setup of Linux, GPU drivers and container runtime,
- hardening of the base installation and preparation for operation,
- fully configured Open WebUI,
- Ollama and/or vLLM selected for the agreed use case,
- the agreed local model installed and technically verified,
- documentation of the delivered configuration,
- initial setup, onboarding and support.
The AI Cube therefore arrives prepared for plug-and-play use. Company-specific data sources, SSO and applications are then connected in a controlled way in the target environment.
Open WebUI, Ollama or vLLM
Open WebUI is the employee-facing workspace. It combines chat, model selection, files, knowledge collections and permissions. A runtime performs model inference:
- Ollama is useful for straightforward model management and moderate usage.
- vLLM provides an OpenAI-compatible API and targets efficient inference and concurrent requests.
The right runtime depends on more than the model name. We consider Arm and GB10 support, model format, throughput and integration requirements.
RAG and company knowledge
GPT-OSS 120B can act as the generation model in a RAG pipeline. A reliable knowledge system needs more than a large model:
- Documents must be extracted, structured and kept current.
- Permissions from source systems must be preserved.
- Embeddings and retrieval determine which content reaches the model.
- Answers should expose citations or source passages.
- Quality must be tested with real questions and expected answers.
Open WebUI includes knowledge features for initial use cases. For large or permission-sensitive collections, we can build custom RAG pipelines, middleware and interfaces.
Limits of the GB10 class
One AI Cube is compact and memory-rich, but it is not an HA GPU cluster. A larger architecture makes sense for:
- many concurrent users with committed response times,
- high availability and defined recovery objectives,
- large models with long contexts,
- high batch throughput,
- training or substantial fine-tuning,
- rack and data centre requirements.
Two AI Cubes can be linked through ConnectX-7. Whether a workload benefits depends on the runtime and parallelisation method.
Decision
GPT-OSS 120B is a plausible option for the AI Cube when a capable local reasoning model is required and the actual workload fits the GB10 class. For some tasks, a smaller model delivers better quality, latency and economics. This is why the model is agreed before delivery rather than fixed for every customer.
Without your own hardware, gpt-oss-120b is available on the AI platform: a dedicated Open WebUI instance with models operated in Germany by German providers.
View the AI Cube and complete delivery scope or discuss the workload.
