Yes — Qwen3-235B-A22B Q5_K_M completed our tests on one GPU with help from the CPU and system RAM. In the controlled benchmark it generated a mean 4.7 tokens/s, or pieces of text per second. We separately ran three short tasks that produced visible final answers.
It is the largest variant tested in this campaign, not a claim about the server’s maximum possible model. Its files occupy 155.4 GiB, exceeding the card’s own memory. Splitting computation makes this configuration possible, but response time and memory behaviour still need assessment.
How can such a large model use one card?
235B means roughly 235 billion parameters — the data describing the model — not 235 GB of memory. This model uses a Mixture of Experts, or MoE: it contains multiple sections called experts, only some of which are selected for each computation step.
That reduces the work performed per step without removing the need to access the remaining model data. Our configuration assigned part of the expert computation to CPU and RAM while the rest used the GPU. A high parameter count should therefore not automatically be read as greater speed or a better answer.
Three examples: answer, time and output format
We checked a simple arithmetic operation, extraction of specified fields from text, and assignment of a category. JSON stores data in fields that an application can read automatically. Each task requested a JSON object alone, without additional formatting.
| Task | First visible text | Completed answer | Plain JSON as requested |
|---|---|---|---|
| Arithmetic | 4.4 s | 7.5 s | No — added Markdown code fence |
| Field extraction from text | 6.5 s | 10.5 s | Yes |
| Category assignment | 6.2 s | 9.0 s | No — added Markdown code fence |
The answer values were correct in all three tasks. One response passed the strict format check: the other two were wrapped in Markdown code fences. That matters for automation, but is not an incorrect calculation or category. We ran each task once; three examples are not a model-quality ranking. A separate response-format review preserves the distinction between correct values and compliance with the requested format.
When does this configuration make sense?
For experimenting with a larger model and queued tasks — yes, if the response time suits the application. Preparing a draft document summary for later review is one possible use. It is a proposed application, not a summarization-quality test performed here.
We would not choose this setup for fluid chat or many simultaneous users without further measurements. One client was active, and longer answers take more time. The short practical tasks in the table are not the 256-token responses used in the speed benchmark.
Assessment: a larger model can genuinely run on one server using GPU and RAM. Access to that model variant is the benefit; generation time, memory requirements and checking output format in the application are the trade-offs. Its usefulness depends on the task, not simply the “235B” label.
See the 72B model compared on GPU and with RAM, see the other tested models or describe your workload. Explore the server configuration in our offer.
Technical details
Model, engine and test machine
We used Qwen3-235B-A22B-GGUF at the pinned revision, in Q5_K_M format, with six weight files totalling 155.4 GiB. This is file size, not a measurement of process memory. The model is released under the Apache 2.0 licence.
| Component | Measurement configuration |
|---|---|
| GPU | NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition, 96 GB VRAM |
| Processor | AMD EPYC 4565P, 16 cores and 32 logical CPUs |
| Test machine system RAM | 96 GB |
| NVIDIA driver | 595.91.07 |
| Engine | llama.cpp 0.5.0-dev, release b11146 |
| Computation split | Expert computation for 44 layers requested on CPU; remaining computation on GPU |
| Synthetic input / output | 2048 or 8192 tokens / 256 tokens |
| Concurrency | One request and one server slot |
| Synthetic repetitions | One separate warm-up followed by three measured responses per input length |
| Practical tasks | Three different tasks, one attempt each; output limit 1024 tokens |
| Context allocation | 8704 tokens; F16 KV cache and Flash Attention |
| CPU / batch / ubatch | 16 threads / 2048 / 512 |
| GPU power limit and cooling | 300 W; automatically controlled chassis fans |
The engine revision was 7fe450e19305b828c199d602c23a8337aaa1f03b. Loading used mmap without mlock; prompt caching, context shifting, automatic memory fitting and lazy mode were disabled. Thinking mode was disabled for this model. The log reports 95/95 layers on GPU, but the separate expert setting routes some computation to CPU — the layer count alone therefore does not mean GPU-only execution. CPU and CUDA buffers are engine allocations, not measurements of resident RAM.
Speed at two input lengths
| Input tokens | First token — median | Complete response — median | Generation — mean | Process CPU — mean share of host capacity |
|---|---|---|---|---|
| 2048 | 16.5 s | 70.6 s | 4.7 tokens/s | 38.1 % |
| 8192 | 51.5 s | 105.6 s | 4.7 tokens/s | 26.6 % |
Each group contains three measured responses of 256 tokens. First-token time in this table refers to the synthetic test stream. For the practical examples above, we time the first visible text of the final answer; different inputs and output lengths prevent direct comparison of these timings.
Generation is the engine’s mean decode rate, not throughput calculated from the complete response time. Release b11146 counts 255 decode steps for 256 output tokens because the first token is produced during input processing. CPU is the process’s mean share of the total capacity of 32 logical CPUs. This tests operation and speed, not the quality of answers produced by the synthetic trials.
How much memory did the model use?
The highest VRAM sample across the combined windows of six synthetic responses was 87.7 GiB. Peak process PSS — resident system RAM with shared areas accounted for proportionally — was 51.5 GiB within those windows. Across the whole case, including loading and warm-up, peak PSS was 80.6 GiB. These are different scopes, not conflicting readings.
We do not add these values to claim that all 155.4 GiB of weights stayed resident in RAM and VRAM. Files were memory-mapped, some data can be served by the system file cache, and preparation involved storage reads. Within the observed intervals of completed synthetic responses, the process counter recorded no bytes read from block storage; however, samples do not cover every interval edge. This does not prove full model residency or general independence from NVMe.
The CSV memory scope combines both input lengths; repeating those maxima in two rows does not create input-specific measurements. PSS is not the whole host’s memory balance, and buffer allocation is not the same as actual resident memory. We also explain RAM and file cache.
Assessment scope and system-memory configuration
The series completed both input lengths and all three practical tasks. Recorded samples showed no thermal throttling; this is not a multi-hour stability test for the model or a measurement of multiple simultaneous clients.
A separate SMBIOS record reports 48 GiB modules in channel A positions A0 and A1, empty B0/B1 slots, and a configured rate of 3600 MT/s. This is a firmware observation, not physical verification or proof of a performance cause. RAM topology and bandwidth matter to CPU execution; we do not use these results to predict the offered 192 GB configuration or another server.
Source results and settings are in the reviewed campaign JSON. The original strict format result remains 1/3. A separate review of the two Markdown-wrapped answers found correct values inside the objects but does not change the full-response parser’s result. It was not used as an automatic correction during the measured run.