Watermarking May Be the Reason World Models Replace LLMs
You Can’t Watermark a Thought - Why Yann LeCun's World Models May Beat LLMs
On Tuesday morning I was doing what I always do before a diligence call, which is re-reading a memo I had already read three times so I could feel prepared instead of being prepared, when someone dropped a TechCrunch link into the group chat:
“Anthropic says it will watermark text generated by its AI models.”
My first reaction was not intellectual. My first reaction was: wait, my memos?
Because, and I say this with the confidence of a man who bills for his judgment, I run half my drafts through Claude. Not to write them. To tighten them. There’s a difference, and I will die on this hill, and apparently the hill is now watermarked.
Somewhere in Brussels, a machine can now detect that my “per my last email” had a co-author.
So that night I did the thing. You know the thing. One tab becomes nine. Nine becomes an arXiv situation. My mum asked what I was doing and I said “research” in the tone of a man watching football.
Here’s what I learned, roughly in the order I learned it.
First: the watermark isn’t hidden in the text. It IS the text. There’s no invisible ink, no secret character. When a language model writes, it’s rolling dice over which word comes next, and the watermark just shaves the dice. Ever so slightly, using a secret key, the model prefers certain words in certain contexts. You can’t see it. I can’t see it. But someone with the key can run the statistics and go: ah. Loaded dice. Claude was here.
Which means the watermark survives copy-paste, because you’re copying the word choices themselves. You’d have to rewrite the sentences to shake it off. Anthropic’s own support page says it “may persist through some editing,” which is the most lawyer-reviewed sentence I’ve read all year, and I structure energy deals for a living.
Second: this has been running quietly for two years and nobody noticed. Google’s DeepMind shipped SynthID-Text inside Gemini back in 2024. They published it in Nature (actual Nature, the dinosaur journal) and reported testing it across some 20 million live responses. Users didn’t notice any drop in quality. Twenty million people took a blind taste test of loaded dice and shrugged.
I found that genuinely unsettling in a way I couldn’t name yet. Hold that thought.
Third: the reason this is suddenly everywhere is not a moral awakening. It’s Article 50 of the EU AI Act, which kicked in on August 2. AI-generated content must be “marked in a machine-readable format and detectable as artificially generated.” Fines up to €15 million or 3% of global turnover. About 190 companies signed the accompanying code of practice: Anthropic, OpenAI, Google, Meta, Microsoft, the whole class photo.
In deal terms: this isn’t ESG. This is covenant compliance.
Now. Here is where I started enjoying myself, because the attack literature is where academics stop being polite.
There is a paper (an actual peer-reviewed ICML paper) called “Watermarks in the Sand.” Boaz Barak’s group at Harvard, proving that “strong” watermarking is mathematically impossible as long as an attacker can do two things: judge whether text is still good, and keep nudging it. Which is... everyone? That’s just a person with a paraphrasing tool. They didn’t publish a critique. They published an obituary with a poetic title.
It gets better. Researchers at ETH Zurich showed you can steal a watermark: query the model enough to reverse-engineer the dice-shaving, then either scrub it off your text or stamp it onto text the model never wrote. Cost of the attack: under fifty dollars. Success rate: over 80%.
Read that second part again. For the price of a mediocre business lunch, you can frame someone. You can take a paragraph a human wrote and make it test positive for AI. In project finance we’d call a $50 attack surface against a €15M penalty regime an asymmetric risk profile, and we would not be smiling when we said it.
And my favourite: watermarks are radioactive. If someone trains their own model on watermarked output, the watermark contaminates the student model. Detectably. The mark is hereditary. Somewhere out there is a bootleg model that caught a watermark the way you catch an accent.
Meanwhile OpenAI (and I had to double-check this) has reportedly had a ~99.9% accurate detector for its own text sitting in a drawer for about two years. Unreleased. Their stated reasons include that paraphrasing beats it anyway and that detectors tend to false-flag non-native English writers, which is the politest way of saying the thing punishes exactly the wrong people.
So by 1am my summary was: everyone is now required to stamp AI text; the stamp is cheap to apply, cheap to remove, cheap to forge; and the companies enforcing it have published the receipts on why it doesn’t fully work. Great. Wonderful. Regulation as performance art.
I was about to close the laptop.
Then I had the small thought that turned into this essay.
Every single thing I’d read — the dice-shaving, the sand, the radioactivity, the fifty-dollar heists — all of it attaches to one place: the output. The words. The pixels. The mouth of the machine.
And I remembered that the most heavily funded contrarian in AI has spent years insisting that the mouth is the wrong part of the machine entirely.
Yann LeCun. The JEPA guy. He left Meta in December, started AMI Labs in Paris, and raised a $1.03 billion seed round (reportedly Europe’s largest ever) to build “world models” instead of language models.
If you haven’t met JEPA, the short version: a language model predicts the next word. A video generator predicts the next pixel. JEPA predicts the next meaning. It watches the world and forecasts what happens next in an abstract representation space, and Meta’s own description of V-JEPA 2 is that it deliberately discards pixel-level detail as unpredictable noise. It doesn’t paint you a picture of the ball landing. It just... knows where the ball lands.
It is, architecturally, the quiet one in the meeting. Sees everything. Says nothing.
And here’s what stopped me: you cannot watermark that.
Not “it’s hard.” Not “the tech isn’t ready.” There is no attachment point. A watermark is a bias in sampled output: words, pixels, audio. JEPA’s output is an embedding. A vector. A thought, basically. I went looking for papers on watermarking JEPA-class models, fully expecting the usual arXiv pile.
Nothing. Zero. The literature on watermarking language models is a library. The literature on watermarking latent world models is an empty shelf with good lighting.
Even the law agrees, accidentally. Article 50 attaches to synthetic audio, image, video, or text. An embedding is none of those. The EU wrote a mandate for everything that talks, and LeCun’s entire billion-dollar bet is machines that don’t talk.
You can’t be quoted in the minutes if you never speak. Every deal lawyer knows this. Apparently LeCun does too.
And that’s when the actual shape of the thing showed up, five hours after the group chat message:
This was never about detection. It’s a tax on speaking.
Everything we call “generative AI” earns its keep by producing content; content is exactly the thing regulators can mandate marks on, professors can run detectors on, and fifty-dollar attackers can forge. The entire compliance apparatus of 2026, the watermarks and the C2PA stamps and the verify-this-upload tools, all of it polices the mouth.
World models are a bet that the value of AI eventually migrates from the mouth to the mind: from producing plausible content to understanding and acting. A robot that stacks boxes correctly leaves no font. A model that predicts your grid’s load curve outputs a decision, not an essay. There’s nothing to stamp, nothing to spoof, nothing for a professor to feed into a detector at 2am.
So here’s my probably-wrong-but-I-can’t-shake-it observation: if JEPA-style systems end up winning the next decade, the benchmark charts will only be half the story. The other half is that they quietly exited the content economy — right as the content economy got a customs office.
Now let me argue with myself, because the research kept me honest on three points.
One: silent models aren’t invisible. There’s a whole parallel literature (SSLGuard, REEF) on fingerprinting models by their internal representations, robust to fine-tuning, pruning, even merging. You can’t watermark a thought, but you can apparently biopsy the brain that thinks it. If someone forks V-JEPA’s open weights and claims it’s theirs, there’s a lab test for that. Different question (who copied whom, not what did it say), but it exists.
Two: the moment a world model grows a mouth (bolts on a video decoder, a language head, anything that renders for humans), it walks straight back into watermark country. NVIDIA’s Cosmos world models generate actual video, and sure enough, those outputs get watermarked. The exemption isn’t for “world models.” It’s for silence.
Three, and most humbling: JEPA cannot tighten my memo. That’s the whole reason I was in Claude on Tuesday. The talking models are taxed because talking is what we currently pay them for. Nobody watermarks the intern who never speaks, but nobody asks him to draft the IC paper either. Yet.
Still. I keep coming back to the picture of it.
One branch of AI now writes with loaded dice, wears a stamp it can’t fully keep and can’t fully trust, gets forged for lunch money, and files compliance paperwork in three jurisdictions.
The other branch watched a million hours of video, formed some opinions about how the world works, and has never once said anything you could hold against it.
I know which one sounds like the senior partner.
Anyway. I wrote this one myself. You’ll have to take my word for it, which, I learned this week, is the only kind of proof any of us has left.
— Kay
Papers, for the fellow sickos: Watermarks in the Sand (ICML ‘24) · Watermark Stealing (ICML ‘24) · SynthID-Text (Nature, ‘24) · Radioactive watermarks (NeurIPS ‘24) · LLM Idiosyncrasies (ICML ‘25): the one showing models are 97% identifiable by their word habits alone, no watermark needed, which possibly deserves its own essay · V-JEPA 2 · REEF (ICLR ‘25).