Fine-tuning Data Efficiency › Implementation
Michael BrenndoerferNovember 28, 202516 min read
Language AI HandbookAll chapters Part 34 of 67Fine-tuning Fundamentals
Fine-tuning Data Efficiency › ImplementationOne section · Sections in this chapter6 / 11 ↓
Sections in this chapter11
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's implement several data efficiency techniques, starting with data augmentation.
In[7]:
Codeimport warnings
warnings.filterwarnings("ignore")
# Sample dataset: sentiment classification
train_data = [
("The food was absolutely delicious and fresh", "positive"),
("Service was slow and the waiter was rude", "negative"),
("Great atmosphere and reasonable prices", "positive"),
("Would not recommend, overpriced for quality", "negative"),
("Best meal I've had in months", "positive"),
("Cold food and long wait times", "negative"),
("Friendly staff and tasty appetizers", "positive"),
("Disappointing experience overall", "negative"),
]
# Vocabulary for synonym replacement
synonyms = {
"great": ["excellent", "wonderful", "fantastic", "amazing"],
"good": ["nice", "fine", "decent", "pleasant"],
"best": ["finest", "top", "greatest", "premier"],
"bad": ["poor", "terrible", "awful", "dreadful"],
"slow": ["sluggish", "unhurried", "delayed", "tardy"],
"friendly": ["welcoming", "warm", "amiable", "cordial"],
"tasty": ["delicious", "flavorful", "scrumptious", "yummy"],
"cold": ["chilly", "cool", "frigid", "icy"],
"long": ["extended", "lengthy", "prolonged", "endless"],
}Now let's implement the basic augmentation techniques.
In[8]:
Codeimport random
def synonym_replacement(text, n_replacements=1):
"""Replace n random words with synonyms."""
words = text.split()
augmented_words = words.copy()
# Find words that have synonyms
replaceable_indices = [
i for i, w in enumerate(words) if w.lower() in synonyms
]
if not replaceable_indices:
return text
# Replace random words
n_to_replace = min(n_replacements, len(replaceable_indices))
indices_to_replace = random.sample(replaceable_indices, n_to_replace)
for idx in indices_to_replace:
word = words[idx].lower()
synonym_list = synonyms[word]
replacement = random.choice(synonym_list)
# Preserve capitalization
if words[idx][0].isupper():
replacement = replacement.capitalize()
augmented_words[idx] = replacement
return " ".join(augmented_words)
def random_deletion(text, p=0.1):
"""Randomly delete words with probability p."""
words = text.split()
# Don't delete if sentence is too short
if len(words) <= 3:
return text
new_words = [w for w in words if random.random() > p]
# Ensure we keep at least some words
if len(new_words) == 0:
return random.choice(words)
return " ".join(new_words)
def random_swap(text, n_swaps=1):
"""Randomly swap n pairs of words."""
words = text.split()
if len(words) < 2:
return text
augmented_words = words.copy()
for _ in range(n_swaps):
idx1, idx2 = random.sample(range(len(augmented_words)), 2)
augmented_words[idx1], augmented_words[idx2] = (
augmented_words[idx2],
augmented_words[idx1],
)
return " ".join(augmented_words)
def random_insertion(text, n_insertions=1):
"""Insert random words from the text at random positions."""
words = text.split()
augmented_words = words.copy()
for _ in range(n_insertions):
word_to_insert = random.choice(words)
insert_position = random.randint(0, len(augmented_words))
augmented_words.insert(insert_position, word_to_insert)
return " ".join(augmented_words)Let's see these augmentation techniques in action.
In[9]:
Codeoriginal = "Great atmosphere and reasonable prices"
# Generate examples for each technique
syn_examples = [synonym_replacement(original) for _ in range(3)]
del_examples = [random_deletion(original, p=0.2) for _ in range(3)]
swap_examples = [random_swap(original) for _ in range(3)]
ins_examples = [random_insertion(original) for _ in range(3)]Out[10]:
ConsoleOriginal: Great atmosphere and reasonable prices Synonym replacement: → Excellent atmosphere and reasonable prices → Wonderful atmosphere and reasonable prices → Wonderful atmosphere and reasonable prices Random deletion (p=0.2): → Great atmosphere and prices → atmosphere and → Great atmosphere and reasonable prices Random swap: → atmosphere Great and reasonable prices → Great atmosphere reasonable and prices → Great and atmosphere reasonable prices Random insertion: → Great atmosphere atmosphere and reasonable prices → Great Great atmosphere and reasonable prices → reasonable Great atmosphere and reasonable prices
These examples show how each technique modifies the input. Synonym replacement and random insertion introduce lexical variety, while random swap and deletion force the model to be reliable to noise and structural changes. Note that while meaning is largely preserved, some fluency is lost, which is a trade-off in text augmentation.
Now let's implement a complete augmentation pipeline that combines these techniques.
In[11]:
Codeclass EDAugmenter:
"""Easy Data Augmentation (EDA) for text.
Combines four simple augmentation operations:
- Synonym replacement
- Random insertion
- Random swap
- Random deletion
"""
def __init__(
self, synonyms_dict, alpha_sr=0.1, alpha_ri=0.1, alpha_rs=0.1, p_rd=0.1
):
self.synonyms = synonyms_dict
self.alpha_sr = alpha_sr # % of words to replace with synonyms
self.alpha_ri = alpha_ri # % of words to insert
self.alpha_rs = alpha_rs # % of words to swap
self.p_rd = p_rd # probability of random deletion
def augment(self, text, num_aug=4):
"""Generate multiple augmented versions of a text."""
words = text.split()
n = len(words)
augmented_texts = []
for _ in range(num_aug):
aug_text = text
# Apply each technique with some probability
if random.random() < 0.5:
n_sr = max(1, int(self.alpha_sr * n))
aug_text = synonym_replacement(aug_text, n_sr)
if random.random() < 0.5:
n_ri = max(1, int(self.alpha_ri * n))
aug_text = random_insertion(aug_text, n_ri)
if random.random() < 0.5:
n_rs = max(1, int(self.alpha_rs * n))
aug_text = random_swap(aug_text, n_rs)
if random.random() < 0.5:
aug_text = random_deletion(aug_text, self.p_rd)
augmented_texts.append(aug_text)
return augmented_texts
def augment_dataset(self, data, num_aug_per_example=4):
"""Augment an entire dataset."""
augmented_data = []
for text, label in data:
# Keep original
augmented_data.append((text, label))
# Add augmented versions
for aug_text in self.augment(text, num_aug_per_example):
augmented_data.append((aug_text, label))
return augmented_dataLet's augment our small training set and see the expansion.
In[12]:
Codeaugmenter = EDAugmenter(synonyms)
augmented_data = augmenter.augment_dataset(train_data, num_aug_per_example=3)Out[13]:
ConsoleOriginal dataset size: 8 Augmented dataset size: 32 Expansion factor: 4.0x --- Example augmentations --- Original: The food was absolutely delicious and fresh Augmented versions: → was The food was absolutely delicious and fresh → The food was absolutely delicious delicious and fresh → The food absolutely delicious and fresh
The dataset size has expanded by a factor of four. This gives materially more training signal. The examples show that the augmented versions retain the core sentiment of the original ("positive" or "negative") while introducing variations in phrasing, which helps prevent overfitting to specific words.
Implementing SetFit-Style Training
Now let's implement the core idea behind SetFit: generating contrastive pairs from few-shot data.
In[14]:
Codefrom itertools import combinations
def generate_contrastive_pairs(data, num_positive=10, num_negative=10):
"""Generate positive and negative pairs for contrastive learning."""
# Group examples by label
label_to_examples = {}
for text, label in data:
if label not in label_to_examples:
label_to_examples[label] = []
label_to_examples[label].append(text)
positive_pairs = []
negative_pairs = []
# Generate positive pairs (same label)
for label, examples in label_to_examples.items():
pairs = list(combinations(examples, 2))
positive_pairs.extend(pairs[:num_positive])
# Generate negative pairs (different labels)
labels = list(label_to_examples.keys())
for i, label1 in enumerate(labels):
for label2 in labels[i + 1 :]:
for ex1 in label_to_examples[label1]:
for ex2 in label_to_examples[label2]:
negative_pairs.append((ex1, ex2))
if len(negative_pairs) >= num_negative * len(labels):
break
return positive_pairs, negative_pairs
positive_pairs, negative_pairs = generate_contrastive_pairs(train_data)Out[15]:
ConsoleGenerated 12 positive pairs Generated 16 negative pairs --- Example positive pairs (same sentiment) --- Text 1: The food was absolutely delicious and fresh... Text 2: Great atmosphere and reasonable prices... Text 1: The food was absolutely delicious and fresh... Text 2: Best meal I've had in months... Text 1: The food was absolutely delicious and fresh... Text 2: Friendly staff and tasty appetizers... --- Example negative pairs (different sentiment) --- Text 1: The food was absolutely delicious and fresh... Text 2: Service was slow and the waiter was rude... Text 1: The food was absolutely delicious and fresh... Text 2: Would not recommend, overpriced for quality... Text 1: The food was absolutely delicious and fresh... Text 2: Cold food and long wait times...
This shows how few-shot data can be amplified. From just 8 examples (4 per class), we generated many training pairs for contrastive learning.
Few-Shot Fine-Tuning with Sentence Transformers
Let's implement few-shot fine-tuning using the SetFit approach with a pre-trained sentence transformer.
In[16]:
Codefrom sentence_transformers import SentenceTransformer
# Load a pre-trained sentence transformer
model_name = "all-MiniLM-L6-v2"
encoder = SentenceTransformer(model_name)
# Our "few-shot" training data
train_texts = [text for text, _ in train_data]
train_labels = [1 if label == "positive" else 0 for _, label in train_data]
# A larger test set
test_data = [
("Amazing food, will definitely come back", "positive"),
("Terrible service, never again", "negative"),
("The pasta was perfectly cooked", "positive"),
("Way too expensive for what you get", "negative"),
("Lovely ambiance and great cocktails", "positive"),
("Found a hair in my soup", "negative"),
("Highly recommend the dessert menu", "positive"),
("Staff seemed annoyed by questions", "negative"),
("Fresh ingredients and creative dishes", "positive"),
("Waited 45 minutes for appetizers", "negative"),
]
test_texts = [text for text, _ in test_data]
test_labels = [1 if label == "positive" else 0 for _, label in test_data]First, let's see how well the pre-trained embeddings work without any fine-tuning.
In[17]:
Codefrom sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# Embed training and test data
train_embeddings = encoder.encode(train_texts)
test_embeddings = encoder.encode(test_texts)
# Train a simple classifier on the embeddings
classifier = LogisticRegression(max_iter=1000)
classifier.fit(train_embeddings, train_labels)
# Evaluate
predictions = classifier.predict(test_embeddings)
accuracy = accuracy_score(test_labels, predictions)Out[18]:
ConsoleFew-shot accuracy (8 training examples): 1.000
Classification Report:
precision recall f1-score support
negative 1.00 1.00 1.00 5
positive 1.00 1.00 1.00 5
accuracy 1.00 10
macro avg 1.00 1.00 1.00 10
weighted avg 1.00 1.00 1.00 10The baseline performance shows that pre-trained sentence embeddings capture significant semantic information even without task-specific tuning. Achieving accuracy well above random guessing (50%) with just 8 examples confirms the value of transfer learning.
Now let's compare with augmented training data.
In[19]:
Code# Augment training data
augmented = augmenter.augment_dataset(train_data, num_aug_per_example=4)
aug_texts = [text for text, _ in augmented]
aug_labels = [1 if label == "positive" else 0 for _, label in augmented]
# Embed augmented data
aug_embeddings = encoder.encode(aug_texts)
# Train classifier on augmented data
classifier_aug = LogisticRegression(max_iter=1000)
classifier_aug.fit(aug_embeddings, aug_labels)
# Evaluate
predictions_aug = classifier_aug.predict(test_embeddings)
accuracy_aug = accuracy_score(test_labels, predictions_aug)Out[20]:
ConsoleOriginal training size: 8 Augmented training size: 40 Few-shot accuracy (original): 1.000 Few-shot accuracy (augmented): 1.000
Data augmentation typically yields a performance improvement by helping the model generalize from the limited examples. By seeing variations in phrasing and word choice, the classifier becomes less reliant on specific keywords and more focused on the underlying sentiment.
Contrastive Fine-Tuning Implementation
For a more advanced approach, let's implement contrastive fine-tuning on the sentence embeddings.
In[21]:
Codeimport torch
import torch.nn as nn
import torch.nn.functional as F
class ContrastiveLoss(nn.Module):
"""Contrastive loss for learning sentence embeddings."""
def __init__(self, margin=0.5):
super().__init__()
self.margin = margin
def forward(self, emb1, emb2, label):
"""
emb1, emb2: embeddings of the pair
label: 1 for positive pair, 0 for negative pair
"""
# Cosine similarity
cos_sim = F.cosine_similarity(emb1, emb2)
# Contrastive loss:
# Positive pairs: minimize (1 - similarity)
# Negative pairs: maximize similarity beyond margin
loss_positive = label * (1 - cos_sim)
loss_negative = (1 - label) * torch.clamp(cos_sim - self.margin, min=0)
return (loss_positive + loss_negative).mean()
def train_contrastive_epoch(encoder, pairs, labels, optimizer, loss_fn):
"""Train one epoch of contrastive learning."""
encoder.train()
total_loss = 0
# Batch the pairs for efficiency
texts1 = [p[0] for p in pairs]
texts2 = [p[1] for p in pairs]
# Get embeddings (in practice, you'd do this in batches)
with torch.no_grad():
emb1 = torch.tensor(encoder.encode(texts1))
emb2 = torch.tensor(encoder.encode(texts2))
labels_tensor = torch.tensor(labels, dtype=torch.float32)
loss = loss_fn(emb1, emb2, labels_tensor)
total_loss = loss.item()
return total_lossLet's visualize how the embeddings cluster before any task-specific training.
In[22]:
Codefrom sklearn.decomposition import PCA
# Combine train and test for visualization
all_texts = train_texts + test_texts
all_labels = train_labels + test_labels
all_embeddings = encoder.encode(all_texts)
# Reduce to 2D for visualization
pca = PCA(n_components=2)
embeddings_2d = pca.fit_transform(all_embeddings)Out[23]:
Visualization
The visualization shows that pre-trained sentence embeddings already capture sentiment distinctions. The positive and negative examples form relatively distinct clusters even without any fine-tuning. This is why few-shot learning works: the pre-trained model has already learned real representations that transfer to new tasks.
Now let's visualize how data augmentation affects the embedding space by comparing original and augmented embeddings.
Out[24]:
Visualization
Analyzing Sample Efficiency
Let's examine how performance scales with training data size.
In[25]:
Codedef evaluate_sample_efficiency(
encoder,
train_data,
test_texts,
test_labels,
samples_per_class=[1, 2, 4, 8, 16],
):
"""Evaluate accuracy at different training set sizes."""
results = []
pos_pool = [(t, l) for t, l in train_data if l == "positive"]
neg_pool = [(t, l) for t, l in train_data if l == "negative"]
for n_samples in samples_per_class:
# Train and evaluate multiple times, sampling different subsets each run
accuracies = []
for seed in range(10): # More runs for stable variance estimates
random.seed(seed)
sampled_pos = random.sample(pos_pool, min(n_samples, len(pos_pool)))
sampled_neg = random.sample(neg_pool, min(n_samples, len(neg_pool)))
sampled_data = sampled_pos + sampled_neg
texts = [t for t, _ in sampled_data]
labels = [1 if l == "positive" else 0 for _, l in sampled_data]
embeddings = encoder.encode(texts)
clf = LogisticRegression(max_iter=1000, C=1.0)
clf.fit(embeddings, labels)
test_embeddings_local = encoder.encode(test_texts)
preds = clf.predict(test_embeddings_local)
accuracies.append(accuracy_score(test_labels, preds))
results.append(
{
"samples_per_class": n_samples,
"total_samples": n_samples * 2,
"mean_accuracy": np.mean(accuracies),
"std_accuracy": np.std(accuracies),
}
)
return results
# Use augmented data as the training pool so higher n values draw genuinely
# varied examples rather than duplicates
extended_train_data = augmented_data # 40 augmented examples (8 orig × 5)
results = evaluate_sample_efficiency(
encoder,
extended_train_data,
test_texts,
test_labels,
samples_per_class=[1, 2, 4, 8, 16],
)Out[26]:
Visualization
Out[27]:
ConsoleSample Efficiency Results: -------------------------------------------------- Samples per class: 1 | Total: 2 | Accuracy: 0.680 ± 0.125 Samples per class: 2 | Total: 4 | Accuracy: 0.760 ± 0.136 Samples per class: 4 | Total: 8 | Accuracy: 0.840 ± 0.080 Samples per class: 8 | Total: 16 | Accuracy: 0.890 ± 0.083 Samples per class: 16 | Total: 32 | Accuracy: 0.900 ± 0.000
The sample efficiency curve reveals several important patterns. First, the steepest improvements come from the first few examples. Going from 1 to 2 examples per class often gives a larger accuracy gain than going from 4 to 8. Second, the broad pattern is that variance falls as the training pool becomes more representative, but a ten-run estimate remains noisy at the smallest sample sizes. Here the four-sample condition varies slightly more than the two-sample condition, and the eight- and sixteen-sample estimates are nearly tied. Few-shot learning with just 1-2 examples per class still shows high run-to-run variance, making results less reliable. Third, pre-trained models make the curve much steeper than randomly initialized models would show. The model already understands language; it just needs a few examples to calibrate for the specific task.
Out[28]:
Visualization