Repetition Penalties › When to Use Each Penalty

The choice between penalty types depends on your generation task. Each technique has a domain where it performs best, and using the wrong technique can introduce problems while failing to solve the ones you have.

Repetition Penalty works well as a general-purpose solution. It's the most widely implemented (available in Hugging Face's generate() method) and handles most cases adequately. Use it when you want a simple, effective baseline for reducing repetition, particularly when you're not sure which penalty type is most appropriate. The single-parameter interface makes it easy to tune and reason about. Typical starting value: $\theta = 1.1$ to $1.2$.

Frequency Penalty excels when you want to allow some repetition while preventing excessive use. It's particularly useful for:

• Long-form content where occasional word repetition is acceptable but word overuse is not
• Creative writing where you want natural variation without harsh constraints
• Technical writing where certain terms must repeat but shouldn't dominate

Presence Penalty promotes vocabulary diversity and topic breadth. Consider it for:

• Brainstorming and idea generation where you want many distinct concepts
• Summarization where you want to cover multiple points without redundancy
• Conversational agents that should avoid fixating on specific words

N-gram Blocking provides guaranteed protection against exact repetition. Use it when:

• Exact phrase repetition would be clearly wrong (e.g., safety-critical applications)
• You need deterministic behavior rather than probabilistic reduction
• Other penalties aren't sufficiently preventing loops

For many applications, combining moderate repetition penalty ($\theta \approx 1.1$-$1.2$) with nucleus sampling provides a good balance between preventing loops and maintaining fluent output.