A large model file, a process's memory footprint and the system's “used” memory figure do not describe the same thing. Linux can cache data it has read, the application can map the model file, and model layers can also reside in VRAM.
Sizing RAM therefore means tracking system headroom and process memory, not just the size of a GGUF file. We cover GPU capacity separately in our guide to models that fit in 96 GB VRAM.
Our Qwen3-8B memory measurement
In short runs with the model on the GPU, the highest recorded process PSS was 813.20 MiB. This is memory attributed to the process by a particular counter, not the server's total RAM requirement. The operating system, file cache and other services also use memory.
| Phase | Highest process PSS reading | Lowest system MemAvailable | Samples |
|---|---|---|---|
| Model ready, 10 s without requests immediately after loading | 750.02 MiB | 89.41 GiB | 5 |
| 2048 input tokens, 1 concurrent request | 782.44 MiB | 89.39 GiB | 3 |
| 8192 input tokens, 4 concurrent requests | 813.20 MiB | 89.35 GiB | 10 |
MiB and GiB are binary units: 1 GiB = 1024 MiB. The chart shows the same peak PSS readings in GiB. Swap use was zero in samples from these phases. The system also retained file cache from earlier work; we do not attribute all of that cache to Qwen3-8B.
Hardware and method
| Component | Measurement configuration |
|---|---|
| System RAM | 96 GB |
| CPU | AMD EPYC 4565P, 16 cores / 32 threads |
| GPU | RTX PRO 6000 Blackwell Max-Q Workstation Edition, 96 GB VRAM |
| Operating system and driver | Rocky Linux 10.2, NVIDIA 595.91.07 |
| Model and engine | Qwen3-8B Q4_K_M, llama.cpp b11146, GPU-resident model |
Each scenario had a warmup and three measured rounds, with 256 new tokens per response. The local API had four 8704-token slots, F16 KV cache and Flash Attention; prompt caching was disabled. Before each scenario, the GPU cooled to no higher than 40°C.
Process RSS/PSS and host memory were read through /proc roughly every two seconds. The table includes ready-idle samples and samples within completed measured rounds, excluding warmup and gaps. Recorded maxima can miss shorter spikes. Model loading had only one sample, so we do not use it to establish peak memory use.
The operating system cache was warm after verifying the model checksum. Available RAM was not restricted, so this is not a minimum-capacity test. CSV also contains peak RSS readings, while JSON includes ranges and counter definitions. This telemetry's overhead distinguishes the study from the earlier API throughput test.
Three readings, three different questions
| Reading | What does it tell you? |
|---|---|
| MemAvailable | How much memory can the system make available to new applications without swapping? |
| Process RSS | How much of the process's mapped memory currently resides in RAM? |
| Process PSS | How much memory belongs to the process after shared pages are divided proportionally? |
MemAvailable includes some reclaimable memory, so it is not the same as MemFree. Shared data can appear in the RSS of multiple processes; PSS apportions that share. See the Linux /proc memory documentation.
Do not simply add process RSS to the entire system cache as two independent model costs. Those readings can partly describe the same memory pages.
Why a second startup can be different
After a model file is read, some data may remain in the operating system's cache. A later launch can then use memory rather than reading the entire file from storage again. This is a warm start, unlike a start without that data cached.
Even a checksum verification reads the entire file and can warm the cache before a measurement. Report the conditions alongside the time between launching the program and API readiness. A quick second launch is not, by itself, an NVMe throughput test.
llama.cpp offers different model loading modes, including memory mapping. File size does not imply an identical amount of private process memory in every mode.
How to establish RAM headroom
Observe model loading, ready-idle operation and request handling separately. Include the services that will actually run alongside the model: a document database, index, API, job queue or file processing.
One useful signal is the lowest observed MemAvailable during a representative task, considered alongside swap and response time. Allow headroom for model updates, other processes and short workload spikes too. A single short run does not establish minimum RAM for every application using that model.
When more RAM can help
More system memory provides room for additional services and cache, and for host-side model data in configurations that partly use the CPU. It does not automatically become GPU VRAM or guarantee more tokens per second when the bottleneck is elsewhere.
Before expanding memory, identify what is actually constrained: host RAM, GPU capacity, CPU time or data access. This helps size the server for the application rather than a single specification number.
Related reading: CPU work during GPU inference and choosing a Linux distribution.
View the server configuration or describe the models and services you want to run together.