ROUGE Scores › ROUGE-W: Weighted Longest Common Subsequence

Standard LCS treats all matching subsequences equally, regardless of whether the matches are consecutive or scattered. However, consecutive matches are linguistically more significant than scattered ones. "The quick brown fox" is a better match than "The fox quick brown" even if both have the same LCS length with a reference. The first preserves syntactic structure and readability, while the second is jumbled and confusing.

This limitation of standard LCS becomes apparent with a concrete example. Suppose the reference is "The committee approved the budget proposal after extended debate." Candidate A reads "The committee approved the proposal after debate" (two words omitted, but contiguous phrasing preserved). Candidate B reads "The committee debate after budget approved the proposal" (all words present but scrambled). Both candidates might reach the same LCS length relative to the reference, because LCS allows arbitrary gaps. But Candidate A is clearly better: it reads naturally and conveys the right meaning. Candidate B is in effect incomprehensible. Standard ROUGE-L would not distinguish between them.

ROUGE-W addresses this by introducing a weighting function that favors consecutive matches. The weighted LCS (WLCS) applies a penalty to gaps in the matching sequence. This keeps fragmented matches contribute less to the final score than cohesive blocks of text.

The standard ROUGE-W implementation uses a weight function that grows faster than linear with consecutive matches. For a consecutive sequence of length $k$, the weight is often computed as:

$$w(k) = k^\alpha$$

where:

• $k$: the length of a consecutive matching sequence
• $\alpha$: an exponent greater than 1 (typically $\alpha = 1.2$ or $2$), so consecutive matches contribute disproportionately more to the score than scattered matches

For example, with $\alpha = 2$:

• A consecutive match of 4 words contributes $4^2 = 16$ to the score
• Four scattered single-word matches contribute $4 \times 1^2 = 4$ to the score

The superlinear weighting means that keeping matches together is worth far more than spreading them out. A system that produces a two-word consecutive match and another two-word consecutive match scores $2^2 + 2^2 = 8$, whereas the same four words scattered individually score $4 \times 1 = 4$. This creates an incentive for the evaluated summary to use reference phrasing in cohesive chunks rather than picking up scattered words.

The WLCS dynamic programming algorithm modifies the standard LCS recurrence to account for these weights, favoring paths that keep matches consecutive. The key modification is tracking the LCS length and the current consecutive run length. When calculating the optimal subsequence, the algorithm must now consider whether words match and whether continuing a consecutive sequence yields higher weight than starting a new scattered match. The recurrence becomes:

$$\text{WLCS}(i, j) = \begin{cases} \text{WLCS}(i-1, j-1) + w(c(i,j)) & \text{if } x_i = y_j \\ \max(\text{WLCS}(i-1, j),\ \text{WLCS}(i, j-1)) & \text{otherwise} \end{cases}$$

where $c(i,j)$ is the length of the current consecutive run ending at positions $i$ and $j$. The algorithm tracks this run length alongside the WLCS value, incrementing it on diagonal moves and resetting it on horizontal or vertical moves.

ROUGE-W is particularly useful in several situations. It shines when fluency matters: you want to penalize summaries that have the right words but in jumbled order. In applications like headline generation or keyphrase extraction, where the output must read naturally, ROUGE-W gives better discrimination than standard ROUGE-L.

ROUGE-W is also appropriate for extractive summarization evaluation. When the system selects sentences from the source, consecutive matches indicate proper sentence extraction. If a system extracts a full sentence verbatim, ROUGE-W rewards it heavily; if it extracts scattered fragments, it receives a lower score despite potentially having the same LCS length. This makes ROUGE-W align better with human judgments in extractive settings.

For short summaries such as single-sentence summaries, word order carries more semantic weight. In longer documents, a few misplaced words might not obscure the meaning, but in a one-sentence summary, "Dogs eat cats" versus "Cats eat dogs" is a catastrophic error that ROUGE-W catches better than ROUGE-L.

However, ROUGE-W is less commonly used than ROUGE-1, ROUGE-2, and ROUGE-L in standard benchmark reporting, partly because the weighting parameter $\alpha$ requires tuning and complicates comparisons across studies. Without standardized values for $\alpha$, scores become incomparable between papers, undermining the metric's utility as a universal benchmark. The practical field has largely settled on ROUGE-1, ROUGE-2, and ROUGE-L as the canonical trio, with ROUGE-W appearing primarily in specialized studies.