Claude's watermark: statistical word choice & a quick check

The term suggests a physical feature like the one on a banknote, visible to the naked eye. Anthropic describes the opposite: the watermark is entirely indistinguishable for readers and changes neither the meaning nor the quality of the text.

There is no string to search for, no suspicious spacing to measure and nothing a manual inspection could surface. The actual mechanism manipulates the source of randomness the model uses to pick between equivalent candidate words — «fast», «quick» and «rapid» in a context where all three mean the same thing.

«AI watermark» is an umbrella term for three technically unrelated levels. Mixing them up means looking in the wrong place.

The checker above covers traces 1 and 3 verifiably, and trace 2 only as a method, because Anthropic's key is not public. File metadata is out of scope here and is handled by the text-humanizer skill.

Anthropic confirms that Claude's approach is a variant of Google DeepMind's SynthID Text, which traces back to a design idea proposed by Scott Aaronson in 2022. This family of methods, which also includes the KGW method published in 2023 (Kirchenbauer, Geiping, Wen et al.), works as follows:

1. 01At every generation step the model holds several candidate tokens with almost identical probability.
2. 02A secret key is combined with the preceding tokens and hashed pseudorandomly to split the entire vocabulary into a green list and a red list.
3. 03Green-list tokens receive a small positive bonus on their selection probability before the model draws the next token.
4. 04That bonus only takes effect where candidates were near-equivalent anyway; it never overrides a strong model preference, which is why grammar and factual content stay unchanged.
5. 05Across a long text the bias accumulates into a statistically measurable pattern, even though every individual word choice looks unremarkable on its own.

Formally, a watermark-free model samples the next token w_t from a conditional distribution P_t over the vocabulary W. A watermarked model additionally couples that choice to a pseudorandom variable ζ_t known only to the verifier, so that w_t = S(P_t, ζ_t), where S is a deterministic decoder function.

Detection runs in reverse: with the same secret key the verifier applies the same hashing scheme to the text under review, counts the share of tokens falling into the green list and compares that share with the value expected under pure chance, typically 50 percent at γ = 0.5. If the observed share deviates significantly upwards, the text is considered likely Claude-generated. The test works without access to the model, but it strictly requires the key.

A positive hit only proves that Claude was involved in producing the text, not who supplied the content or the ideas. A self-written text that Claude merely proofread or translated can trigger exactly the same mark as a fully AI-generated one. Conversely, the absence of a mark is no proof of human authorship.

At the time of the announcement Anthropic had not released a public detection API; the company is working on tools for users and third parties without a binding timeline. Without the real Anthropic key the original procedure cannot be rebuilt or verified externally.

Commercial «AI detectors» that already claim to spot Claude text instead rely on generic stylistic analysis: predictable phrasing, uniform sentence rhythm, recurring structure. Michael Wade's analysis explicitly classifies that as a fundamentally different and far less reliable technique. The regulatory driver behind the measure is Article 50(2) of the EU AI Act, which Anthropic formally signed up to via the Code of Practice.

The underlying principle is openly documented and can be rebuilt for your own purposes, for instance to watermark self-hosted LLM pipelines, even though Anthropic's concrete key stays out of reach. The Python module below implements the core logic: pseudorandom green-list generation per preceding token plus key, and a z-test detector that compares the observed hit rate against the chance expectation.

"""
Claude / LLM Text-Watermark Checker (KGW-Stil / Green-Red-List)
-----------------------------------------------------------------
Implementiert das statistische Grundprinzip, das Anthropic fuer
Claude-Wasserzeichen beschreibt (SynthID-Text / KGW-Familie):
Bei jedem Generierungsschritt wird das Vokabular pseudorandom -
gesteuert durch einen Secret Key + die vorherigen Tokens - in eine
"Green List" und eine "Red List" gesplittet. Ein wasserzeichen-
behaftetes Modell bevorzugt statistisch die Green-List-Tokens.
Ohne den Key ist dieser Bias fuer Menschen unsichtbar, aber mit
genuegend Text statistisch nachweisbar (z-Test).

WICHTIG: Ohne Anthropics echten Secret Key kann dieses Skript
Claude-Wasserzeichen NICHT verifizieren. Es demonstriert die
Methodik und kann fuer eigene, selbst erzeugte Wasserzeichen
(z.B. in eigenen LLM-Pipelines) produktiv genutzt werden.
"""

import hashlib
import math
import re
from typing import List, Dict, Optional


def _hash_token(seed_str: str, token: str) -> int:
    return int(hashlib.sha256(f"{seed_str}:{token.lower()}".encode()).hexdigest(), 16)


def get_green_list(prev_token: str, key: str, vocab: List[str], gamma: float = 0.5) -> set:
    """Pseudorandomer Split des Vokabulars in Green/Red, seeded durch prev_token + Key."""
    seed_str = f"{key}:{prev_token.lower()}"
    scored = sorted(vocab, key=lambda w: _hash_token(seed_str, w))
    n_green = int(len(vocab) * gamma)
    return set(scored[:n_green])


def tokenize(text: str) -> List[str]:
    """Einfache Wort-Tokenisierung (fuer Produktionseinsatz: echten LLM-Tokenizer verwenden)."""
    return re.findall(r"[A-Za-zÀ-ÿ]+(?:'[a-z]+)?", text)


def detect_watermark(
    tokens: List[str],
    key: str,
    gamma: float = 0.5,
    vocab: Optional[List[str]] = None,
    z_threshold: float = 4.0,
) -> Dict:
    """
    Fuehrt den z-Test auf der Green-List-Traefferquote durch.

    - gamma: Erwarteter Anteil Green-Tokens unter Nullhypothese (kein Wasserzeichen).
    - z_threshold: Klassischer Schwellenwert (~4.0) fuer "wahrscheinlich wasserzeichenbehaftet".
    """
    if vocab is None:
        vocab = list(set(t.lower() for t in tokens))
    if len(tokens) < 2:
        return {"error": "Zu wenige Tokens fuer verlaessliche Erkennung (mind. ~75-100 empfohlen)."}

    green_hits = 0
    n = 0
    for i in range(1, len(tokens)):
        prev_tok, cur_tok = tokens[i - 1], tokens[i]
        green = get_green_list(prev_tok, key, vocab, gamma)
        if cur_tok.lower() in green:
            green_hits += 1
        n += 1

    observed_frac = green_hits / n
    se = math.sqrt(gamma * (1 - gamma) / n) if n > 0 else 0
    z = (observed_frac - gamma) / se if se > 0 else 0.0
    p_value = 0.5 * math.erfc(z / math.sqrt(2))

    return {
        "tokens_tested": n,
        "green_hits": green_hits,
        "observed_fraction": round(observed_frac, 4),
        "expected_fraction_null": gamma,
        "z_score": round(z, 3),
        "p_value_one_sided": round(p_value, 6),
        "likely_watermarked": z > z_threshold,
        "reliability_note": (
            "Ausreichend Text (>=100 Woerter empfohlen)."
            if n >= 100 else
            "Warnung: Textmenge unter empfohlenem Minimum, Ergebnis unsicher."
        ),
    }


def check_text(text: str, key: str, gamma: float = 0.5) -> Dict:
    """High-Level-Funktion: Text -> Tokenisierung -> Watermark-Check."""
    tokens = tokenize(text)
    return detect_watermark(tokens, key, gamma=gamma)


if __name__ == "__main__":
    # Beispieltext (menschlich verfasst, kein echtes Anthropic-Wasserzeichen)
    demo_key = "beispiel-secret-key"
    demo_text = (
        "Der Regen fiel leise auf das Dach, waehrend die Katze am Fenster sass "
        "und nach draussen schaute. Es war ein ruhiger Nachmittag, an dem nichts "
        "Aussergewoehnliches geschah, ausser dem gleichmaessigen Klang des Wassers."
    )
    result = check_text(demo_text, demo_key)
    for k, v in result.items():
        print(f"{k}: {v}")

For every text the detector returns the number of tokens tested, the green-list hit rate, the z-score and a one-sided p-value. As a rule of thumb from the literature, a z-score above roughly 4 is strong evidence of a watermark, while below 100 words reliability drops sharply. Production use against real models additionally requires the exact tokenizer of the target model instead of naive word tokenisation, the provider's actual secret key, plus knowledge of the gamma parameter and of the context window size used for hash seeding.

On top of that, the module ki_text_evidenz.py is available for download. It is deliberately not a percentage detector but a transparent evidence key: measurable features score 0 to 3 points each, grouped and weighted into an index with a stated reliability. The «typical AI phrasing» mode above uses exactly that logic — minus the token-statistical features, which would require a locally run language model.

If you need the other direction, that is checking and removing provenance marks in your own files, my skill for that lives on skills.fragroger.ai.

• Anthropic

  Primary source for the announcement and the conditions around the watermark.

• Kirchenbauer et al., A Watermark for Large Language Models

  The KGW method: green list, logit bias and the z-test used by the detector above.

• Kirchenbauer et al., On the Reliability of Watermarks

  Robustness against paraphrasing, translation and human editing.

• Sadasivan et al., Can AI-Generated Text be Reliably Detected?

  Theoretical and practical limits of detection, including spoofing.

• Dathathri et al., SynthID-Text (Nature 2024)

  Google's production text watermark using tournament sampling.

• C2PA Specification

  Standard for signed provenance metadata, complementary to statistical watermarks.