Continuous Post-Training (2025) › Training Methodology

LoRA represents an update to a weight matrix $W \in \mathbb{R}^{m \times n}$ with two smaller matrices. Let $A \in \mathbb{R}^{r \times n}$ and $B \in \mathbb{R}^{m \times r}$, where $r \ll \min(m,n)$. The adapted weight is

Only $A$ and $B$ are trained. This can reduce trainable parameter count and storage, although inference cost depends on whether the adapter remains separate or is merged into the base weights.

A practical update cycle has four boundaries:

1. At the data boundary, record where each example came from, why it belongs in the update, and which model version produced any logged response. Remove private or low-quality feedback before training.
2. At the training boundary, pin the base checkpoint and training configuration. Decide whether the update is an adapter, a merged checkpoint, or a full fine-tune.
3. At the evaluation boundary, measure the stated target and rerun regression sets. Human review is useful when a numeric score cannot capture correctness or policy compliance.
4. At the release boundary, assign a version, begin with limited traffic, monitor predefined metrics, and retain a rollback path.

Forgetting controls fit inside the training and evaluation boundaries. Replay exposes the candidate to selected examples from earlier tasks. Elastic Weight Consolidation (EWC) adds a penalty for changing parameters estimated to matter to those tasks. Separate adapters can isolate changes, but combining them may create interference that was absent when each adapter was tested alone.

An accumulating adapter stack needs an explicit policy. A system may select one adapter per request, compose several adapters at inference time, or merge updates into a new checkpoint. Each choice changes latency and storage requirements. It also changes what must be tested: two adapters that pass independently can still conflict when composed.