Watermarking › Limitations and Impact

Watermarking is a promising primitive, but it comes with significant practical constraints that limit how and where it can be deployed. Understanding these constraints is essential for setting appropriate expectations and designing systems that use watermarking appropriately.

The most fundamental limitation is that no watermarking scheme can be both high-quality and provably robust against paraphrase. This is not a failure of current schemes but a theoretical impossibility result. If the watermarked text contains the same semantic content as the original, a sufficiently capable paraphraser can produce text that conveys the same meaning with different tokens. Since the watermark resides in the tokens, not the meaning, a perfect paraphraser would remove it with zero quality loss.

To see why this is fundamental rather than a gap to be closed by better engineering, consider the information theory perspective. A watermark encodes information (the presence of the watermarked generation) into specific token choices. But the meaning of the text is encoded in the joint distribution of tokens, not in any specific realization. There are exponentially many token sequences that express the same meaning. A perfect paraphraser explores this space and finds one that does not carry the watermark signal. As long as meaning is separable from specific token choices, a sufficiently capable semantic manipulator can defeat any token-level watermark.

This concern already applies in practice. Modern LLMs are increasingly capable paraphrasers. A student with access to a frontier model can paraphrase AI-generated text enough to defeat current watermarking schemes while preserving the argument structure and even much of the phrasing. The robustness of any scheme is ultimately bounded by the capability of available paraphrasers, and that capability is increasing rapidly.

Watermarking requires modifying the sampling procedure at inference time. Every deployment of the model must run the watermarked decoding path, which means the watermarking logic must be integrated into every inference server, edge deployment, and API endpoint. In practice, organizations deploy many fine-tuned variants, distilled models, and quantized versions of their base models. Ensuring that all variants carry the watermark is operationally complex, requiring policy enforcement throughout the deployment pipeline.

Fine-tuning on non-watermarked data can also destroy the watermark signal in some schemes. If the fine-tuned model's logit distributions shift significantly from the base model's, the correlation between logit-boosted tokens and the key-derived green lists may break down, reducing TPR. Distillation is even more problematic: a student model trained on watermarked teacher outputs may produce similar text patterns without the statistical signal, because distillation optimizes for output distribution matching, not for reproducing the watermark's bias.

Watermarking is most effective when the model weights are private and the sampling procedure is controlled by the operator. For open-weights models, anyone can download the weights and run the model with their own sampling code, trivially bypassing the watermark injection step. This means watermarking cannot solve the provenance problem for publicly released models, which is precisely the category of models most used for unmonitored generation. A student who downloads Llama and removes the watermarking code from the generation script faces no technical barrier.

This creates an uncomfortable asymmetry: watermarking is easiest to enforce for the large commercial API providers who have the most operational control, but these providers also have the strongest incentives to build trust through other means. Open-source models, which arguably pose the greater regulatory challenge, are the hardest to watermark reliably.

Most watermarking schemes require a minimum number of tokens for reliable detection, typically 200 or more for soft schemes. Short texts (a sentence, a social media post, a code completion, a product description) cannot be reliably attributed. This is a significant limitation because many high-value uses of LLMs produce short outputs. Targeted misinformation, for instance, often takes the form of short, pithy statements that spread virally. A 30-word disinformation headline falls well below the minimum length for any current watermarking scheme. Multi-bit watermarks (which encode a full message rather than a binary flag) face even worse token requirements because they must transmit more bits of information through the same token-choice channel.

Even a very low false positive rate becomes a burden at scale. If watermark detection is used in academic integrity enforcement at a university with 10,000 students submitting one essay each per year, a FPR of 0.001 implies roughly 10 false accusations per year. False positives in this context carry serious consequences for students who did nothing wrong, including academic sanctions that can affect careers. A FPR that seems negligible in a system context (0.1% of submissions) translates to many real people harmed.

This argues for extremely conservative thresholds in high-stakes settings, which in turn requires longer texts for reliable detection. The practical consequence is that watermarking may not be usable for short-form submissions (a 300-word essay) where the stakes are high and the text is too short for reliable detection at a Bonferroni-corrected threshold. Human review of borderline cases is essential, but this reduces the automation benefit and introduces its own biases.

Detection power varies significantly across domains and languages. Code generation presents a particular challenge: many code tokens are highly constrained by syntax (keywords, mandatory operators), leaving little flexibility for the logit boost to influence choices. A Python function body that must start with def, use standard indentation, and call specific APIs has few degrees of freedom for watermarking. Detection power in constrained domains is substantially lower than in free-form prose.

Non-English languages present a related challenge. Watermarking schemes are typically developed and evaluated on English text. The distribution of logit values, the frequency of near-tie token choices, and the correlation structure of consecutive tokens all vary by language. A scheme calibrated to achieve FPR 0.01 on English text may have a different effective FPR on Chinese, Arabic, or synthetic code, potentially leading to incorrect detection decisions. Careful evaluation across languages and domains is necessary before deployment in multilingual systems.

Despite these limitations, watermarking has opened several important research directions and has changed how the field thinks about AI attribution.

The formalization of the problem as hypothesis testing gave the field a rigorous language for comparing schemes. Before this framework, discussions of watermarking tended to be informal and qualitative. The z-test framework made it possible to state precise claims about FPR, TPR, and minimum text length, enabling apples-to-apples comparison between competing schemes. This formalization is itself a contribution, independent of any specific scheme.

The distortion-free watermarking work resolved an apparent impossibility result and showed that the quality-robustness tradeoff is not fundamental. Before Kuditipudi et al. (2023), it was widely assumed that any detectable watermark must distort the output distribution. The distortion-free scheme demonstrated that this assumption was wrong, opening a new design space. The insight that sampling randomness itself can carry information, independently of the sampled values, is a fundamental result that extends beyond watermarking.

The robustness analysis work helped clarify what watermarking can and cannot guarantee, preventing overconfident claims. Early presentations of watermarking sometimes implied that it was a complete solution to AI attribution. The robustness research showed that this was not so, and redirected attention toward the hardest problems: robustness against paraphrase, detection at short text lengths, and deployment in open-weights settings.

Watermarking also sparked related work on other attribution methods. Provenance via fine-tuning fingerprints embeds information in a model's behavior rather than its outputs, so that the model's origin is recoverable from its response to specific probes. Cryptographic commitments allow a model to prove it generated text without revealing the content, using zero-knowledge protocols. Membership inference approaches test whether specific texts were included in training data. These approaches address different threat models and complement watermarking in a complete attribution system. The watermarking literature's careful problem formulation has benefited all of these adjacent areas.

The field continues to evolve rapidly. Multi-bit watermarks that embed a message rather than just a flag are an active research area, with applications to content tracking and digital rights management. Watermarks that survive multiple rounds of AI-assisted editing are under investigation, with approaches ranging from more reliable hash functions to semantic-level encoding. The policy dimension is also developing: questions about when watermarking should be mandatory, who should hold the keys, what legal status watermark-based attributions carry, and how to handle international deployments where key custody crosses jurisdictions are actively debated in regulatory and academic forums.