Fine-tuning Data Efficiency › Key Parameters

The key parameters used in these data efficiency techniques are:

• alpha_sr / alpha_ri / alpha_rs: Percentage of words to change for synonym replacement, random insertion, and random swap. The original EDA paper recommends values around 0.1 (10% of words), but this depends on sentence length. Short sentences should use lower values to avoid generating nonsensical text.
• p_rd: Probability of deleting a word in random deletion. Values between 0.05 and 0.15 work well in practice. Higher values produce more aggressive deletions that can break sentence structure.
• margin: Distance threshold in contrastive loss. Negative pairs are pushed apart until their cosine similarity drops below this margin. A margin of 0.5 is a common starting point, though domain-specific tasks may benefit from tuning this value based on how naturally separable the classes are.
• max_iter: Maximum iterations for the logistic regression solver. Increasing this value ensures convergence but adds computation time. With small datasets, 1,000 iterations is typically sufficient.
• num_aug: Number of augmented versions to generate per original example. Values between 2 and 8 are typical. The original EDA paper found that generating 4 augmented versions per example provided the best balance between expansion and noise introduction.

Notice that these parameters interact. Setting both alpha_sr and alpha_ri high simultaneously can produce examples that have changed so substantially that the label no longer applies. A sentence that was originally a clear positive review might become ambiguous or even negative after heavy simultaneous synonym replacement and random insertion. Conservative combinations of augmentation techniques are generally safer than aggressive application of any single one.