The GPU handles model computation, but the application still needs a processor. Requests must be accepted, data prepared, execution coordinated and responses delivered. Document retrieval, a database and your application's own logic may also run alongside the model.
“The model fits on the GPU” therefore does not mean “the CPU has nothing to do”. Equally, a high percentage reported for one process does not automatically mean the whole server is overloaded.
What we recorded with Qwen3-8B
In both short scenarios, the llama-server process used CPU time equivalent to roughly one logical CPU, or about 3.1% of the combined time available across 32 hardware threads. This observes a process serving a GPU-resident model; it does not establish a deployment's minimum core count.
| Scenario | Process CPU: 100% = 1 logical CPU | Share of whole-machine CPU time | Measured rounds |
|---|---|---|---|
| 2048 input tokens, 1 concurrent request | 99.67% | 3.11% | 3 |
| 8192 input tokens, 4 concurrent requests | 100.27% | 3.13% | 3 |
These are arithmetic means of three rounds. Process readings ranged from 99.42–100.15% in the first scenario and 100.19–100.40% in the second. The counter covers the whole process rather than text preparation alone; we do not separate computation, driver interaction and active waiting here. We did not restrict available CPUs, so the results do not show that allocating a single thread would preserve performance.
Hardware and method
| Component | Measurement configuration |
|---|---|
| CPU | AMD EPYC 4565P, 16 cores / 32 threads |
| GPU | RTX PRO 6000 Blackwell Max-Q Workstation Edition, 96 GB VRAM |
| System RAM | 96 GB |
| Operating system and driver | Rocky Linux 10.2, NVIDIA 595.91.07 |
| Model and engine | Qwen3-8B Q4_K_M, llama.cpp b11146; 16 engine threads, GPU-resident model |
After warmup, we ran three rounds per scenario, generating 256 new tokens per response. The local API had four 8704-token slots, F16 KV cache and Flash Attention; prompt caching was disabled. Before each scenario, the GPU cooled to no higher than 40°C.
CPU time came from /proc counters at the beginning and end of each round. Additional memory and GPU readings were taken roughly every two seconds. The model's operating system cache was warm after checksum verification. The results do not cover a document database or other RAG application components. JSON includes individual rounds and settings; the extra telemetry distinguishes this study from the earlier throughput benchmark.
One GPU, several kinds of CPU work
Separate two cases. In the first, model layers are on the GPU and the CPU handles the surrounding application. In the second, some of the model deliberately stays on the CPU, which then also participates in model computation.
llama.cpp exposes separate thread settings for generation, input processing and HTTP handling, as well as GPU layer placement. These are documented in the llama-server revision used in our tests. A thread setting does not guarantee that the process continuously occupies that many cores.
Why CPU usage can exceed 100%
Two scales are useful. On the process scale, 100% corresponds to the time of one logical CPU; a multithreaded process can exceed 100%. On the whole-machine scale, 100% covers the combined available time of all logical CPUs.
Our EPYC 4565P provides 32 hardware threads. Dividing the process-scale reading by 32 expresses its share of the machine's total CPU time. This is not a count of occupied physical cores: hardware threads share resources, and the operating system can move work between them.
We calculate utilisation from the increase in CPU time over a defined window, rather than a single snapshot. Linux exposes process and system counters through /proc. That lets us align CPU use with the response time of the same request round.
When to investigate the CPU
Look at several signals together. If response latency rises, the GPU has idle gaps and one or more application threads are busy, investigate the CPU-side stage. Data access or another service can also cause delays, so a CPU percentage alone is not a diagnosis.
For document applications, separate retrieval and context preparation from LLM execution. A model-only benchmark does not measure your PDF parser, vector database or external API calls.
How to choose a thread count
Start from a working configuration and change one setting at a time. For the same requests, record time to first token, response duration and CPU use. More threads are not the goal: a faster application with resources left for its other services is.
If all components run on one machine, include them in the final test. Good results from an isolated LLM process are a starting point, not a measurement of the complete system.
Related reading: system RAM and caching for LLM servers and context length and API concurrency.
View the server configuration or describe the application you want to run.