Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

ELECTRA › Performance Comparison

Michael BrenndoerferUpdated July 25, 20253 min read

Language AI HandbookAll chapters

Part 24 of 67BERT and Variants

Chapter 175ELECTRA › Performance Comparison

Reading section 14 of 18 · Performance Comparison · 3 minSection 14 of 18 · 3 min

Section 14 of 18 · 3 min read

ELECTRA › Performance ComparisonOne section · Sections in this chapter14 / 18 ↓

Sections in this chapter18

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

ELECTRA's results on standard benchmarks demonstrate its efficiency. Here's a comparison with BERT and RoBERTa at various scales:

In[39]:

Code

# Approximate results from ELECTRA paper
benchmark_results = {
    "BERT-Small": {"params": 14, "glue_avg": 75.1, "squad_v2": 68.2},
    "ELECTRA-Small": {"params": 14, "glue_avg": 79.0, "squad_v2": 74.8},
    "BERT-Base": {"params": 110, "glue_avg": 82.2, "squad_v2": 76.3},
    "ELECTRA-Base": {"params": 110, "glue_avg": 85.1, "squad_v2": 80.5},
    "BERT-Large": {"params": 335, "glue_avg": 84.0, "squad_v2": 81.9},
    "RoBERTa-Large": {"params": 355, "glue_avg": 88.1, "squad_v2": 86.5},
    "ELECTRA-Large": {"params": 335, "glue_avg": 89.0, "squad_v2": 88.0},
}

Out[40]:

Console

Benchmark Results Comparison:
-----------------------------------------------------------------
Model              Parameters   GLUE Avg     SQuAD v2.0  
-----------------------------------------------------------------
BERT-Small               14M       75.1         68.2
ELECTRA-Small            14M       79.0         74.8
BERT-Base               110M       82.2         76.3
ELECTRA-Base            110M       85.1         80.5
BERT-Large              335M       84.0         81.9
RoBERTa-Large           355M       88.1         86.5
ELECTRA-Large           335M       89.0         88.0

The results reveal ELECTRA's remarkable efficiency. At each parameter budget, ELECTRA outperforms BERT by 3-4 points on GLUE. Most striking is ELECTRA-Small: with just 14M parameters, it achieves 79.0 on GLUE, comparable to BERT-Base at 110M parameters. This 8× parameter reduction with similar performance demonstrates that sample efficiency translates directly to model efficiency.

ELECTRA-Small outperforms BERT-Base despite having 8x fewer parameters. ELECTRA-Base exceeds RoBERTa-Base's performance. ELECTRA-Large achieves new state-of-the-art results on GLUE, exceeding RoBERTa-Large despite slightly fewer parameters.

Out[41]:

Visualization

Grouped bar chart comparing GLUE and SQuAD scores across BERT and ELECTRA models of various sizes. — Benchmark performance comparison. ELECTRA consistently outperforms BERT at each model size. ELECTRA-Small even exceeds BERT-Base performance with 8x fewer parameters. This shows exceptional parameter efficiency.