Memory Augmentation › Key Parameters
Michael BrenndoerferUpdated July 8, 20252 min read
Language AI HandbookAll chapters Part 18 of 67Long Context
Memory Augmentation › Key ParametersOne section · Sections in this chapter9 / 12 ↓
Sections in this chapter12
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
When implementing memory-augmented models, these parameters significantly affect performance and resource usage:
| Parameter | Description | Typical Values |
|---|---|---|
memory_size | Maximum entries the memory can hold. Larger memories enable longer effective context but increase storage and retrieval costs. | 10K - 1M |
k_neighbors | Entries retrieved per query. Higher values provide more context but dilute relevance. | 4-16 (LM), 16-64 (QA) |
key_dim | Dimensionality of memory keys. Larger dimensions enable finer-grained addressing but increase compute. | 64 - 256 |
value_dim | Dimensionality of stored values. Can be larger than key_dim since values carry the actual information. | 768 - 4096 |
compression_ratio | For compressed memory, how many raw entries compress into one summary. Higher ratios save more memory but lose detail. | 4 - 16 |
temperature | Controls sharpness of retrieval attention. Lower values produce focused retrieval, higher values spread attention. | 0.1 - 2.0 |
write_gate_threshold | For gated writing, the importance threshold for storing entries. Higher thresholds mean more selective storage. | 0.3 - 0.7 |