GGUF Format › Key Parameters
Michael BrenndoerferJanuary 15, 20261 min read
Language AI HandbookAll chapters Part 42 of 67Inference Optimization
GGUF Format › Key ParametersOne section · Sections in this chapter13 / 14 ↓
Sections in this chapter14
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for working with GGUF models and llama.cpp:
- n_ctx: context window size, determining the maximum sequence length the model can process and the peak size of the KV cache.
- n_threads: number of CPU threads to use for inference, typically set to the physical core count.
- n_gpu_layers: number of transformer layers to offload to GPU, where 0 means CPU-only and -1 means all layers on GPU.
- n_batch: batch size for prompt processing, affecting memory usage and prefill throughput.
- tensor_split: distribution of layers across multiple GPUs when using multi-GPU setups, expressed as a fraction per device.