PEFT Comparison › Worked Example: Choosing the Right Method for a Medical QA System

Consider a concrete scenario: you need to build a question-answering system for clinical medical records. The system will answer questions about patient history using text from medical notes. Let's walk through the method selection process systematically.

Step 1: Characterize the task. Medical QA is an information extraction task (NLU-dominant), not pure generation. The model needs to identify needed information in context and produce concise, accurate answers. It will also encounter domain-specific terminology not well-represented in general pre-training data.

Step 2: Inventory your constraints. Suppose you have access to a single NVIDIA A100 with 40GB VRAM, 50,000 labeled question-answer pairs from clinical records, and strict latency requirements of under 200ms per query in production. You cannot afford any inference overhead.

Step 3: Apply the decision framework. Memory is not a constraint (40GB A100 handles standard LoRA on 7B models easily), so QLoRA is not necessary. Inference latency is necessary (under 200ms with zero overhead), which eliminates Adapters. Data is plentiful (50,000 examples), so Prompt Tuning's data-hungry nature is less of a concern, but LoRA's superior performance is still preferred. The task requires domain adaptation (medical terminology), suggesting higher rank is warranted.

Step 4: Make the decision. Use LoRA with $r=32$ applied to all attention projections and the FFN layers, with $\alpha=64$. Merge the weights after training for zero inference overhead. The $r=32$ rank gives sufficient capacity for domain adaptation, while the FFN adaptation helps with the specialized vocabulary. After fine-tuning, merge the LoRA weights into the base model to reach the latency requirement.

Step 5: Plan the validation. Compare the LoRA model against a full fine-tuning baseline on a held-out test set. If the gap exceeds 3% on clinical QA accuracy, consider whether the additional compute of full fine-tuning justifies the performance gain. If the gap is under 2%, the LoRA approach is validated as production-ready.

This worked example illustrates how the abstract decision framework translates into concrete choices. The key is working through constraints in order (hardware, then deployment, then data, then performance) and understanding which constraints eliminate which methods.