11.5 Detection, Watermarks, and Provenance Records

Article 50(2) puts a marking obligation on the model provider, and chapter 11.1 told you what that means: you cannot comply with it, cannot rely on it, and cannot count on it surviving a repost.

This chapter is about the three questions that follow.

1. Can anyone tell my content was generated? Yes, with a confidence score rather than a verdict — and sometimes they get it wrong about content that is not generated at all.
2. Can I prove what I made and how? Only with records you control, because the technical marks are not durable.
3. What do I do when a classifier is wrong about me? There is a procedure, and it is faster if you prepared for it before it happened.

How detection actually works

Three mechanisms are in play, and they fail in different ways.

Classifier models. Providers like Hive offer an AI-generated and deepfake detection API that classifies image, video, and audio and returns a confidence score. Detection services of this kind are widely reported to sit inside platform moderation pipelines. [UNVERIFIED] What matters for you is the shape of the output: a probability, not a fact. A 0.94 is not a determination, and it is the number a platform's review team will treat as one unless you give them something better.

Pixel and latent watermarks. Google's SynthID is embedded in the pixels or the latent representation and is designed to survive crops, compression, and filters. It is detected by Google's own tooling, which is also its limitation: it is not a general-purpose standard. Open alternatives exist — TrustMark is the best-known — and separate commercial watermarking services sell robustness against crop and recompress as their feature.

Provenance manifests. C2PA Content Credentials wrap a signed manifest around a file describing what created it and what was done to it. Camera manufacturers ship it, some generators attach it, and cooperating platforms read and display it.

All three are probabilistic or fragile. None of them is a guarantee, and treating any of them as one is how an operator gets caught out on a Tuesday.

Why the metadata does not survive, and what that costs you

Every reupload path strips metadata: uploading to a platform that re-encodes, screenshotting, screen-recording, downloading through a third-party tool, running a file through an editor, even some CDN configurations. The C2PA manifest attached by your generator is gone after the first platform re-encode, and gone permanently after the first screenshot.

This has one consequence that reorganizes the whole chapter: technical marking is evidence about the first publication, not about every copy in circulation.

So the marks are worth something — you should have them optimally configured — but they cannot be the basis of your compliance story. What they give you:

• Attribution on platforms that read manifests, for the first upload.
• A detection signal that helps platforms label your content correctly instead of guessing.
• A record you can point to if someone claims they generated your work.

What they do not give you: any control over copies in circulation, any ability to prove a downstream file is yours, and any protection against someone deliberately removing the mark.

The legal gap nobody closes

Here is the uncomfortable version of the marking obligation. The AI Act requires providers to mark output. It does not prohibit third parties from removing those marks. There is no provision that makes stripping a watermark or a C2PA manifest an offense, and no liability attached to it in the Act.

The transparency code of practice encourages provenance preservation. Encouragement is not enforcement.

The practical implications:

• "Marked" is not "protected." A marked file can be stripped and reposted with the mark gone, and the person who did it faces no AI Act consequence.
• Do not build a detection strategy on vendor marking. Chapter 7.5's portability rule applies to your evidence as much as to your models.
• Your own independent provenance record is the only durable artifact. It is also the only one under your control, because it lives in your storage rather than in someone's pipeline.

The provenance record you actually keep

One directory per persona, one subdirectory per published asset, and a manifest alongside each file. Minimum viable version:

Two operating rules turn that table into something useful:

Keep the rendered original, not a re-encoded copy. Re-encoding destroys pixel watermarks. Archive the file the generator produced, before any editor or platform touches it.

Write the record at generation time. A provenance manifest reconstructed from memory after a dispute is a narrative. One written at the moment of creation is evidence. If your pipeline is already automated — chapter 7.3 — the record is a byproduct of the run, not a separate chore.

Run a survival test once, per platform. Upload a marked asset, download it back, and run it through the same detector or watermark reader. You will learn within an hour which platforms preserve pixel marks, which strip them, and which re-encode aggressively. Do it before you depend on the answer. [UNTESTED] — this is a procedure, not a result; the outcome depends on each platform's current pipeline and changes without notice.

When the classifier is wrong about you

False positives are not hypothetical. Authentic photographs have been labeled AI-generated, and for a persona account the case is worse than for a photographer: if a real human collaborator, model, or photographer appears in your content, their real image can get labeled, and the label does not distinguish them from you.

The damage is not the label itself. It is the reach that disappears while the label stands.

The procedure, in order:

1. Do not delete the post. Deletion loses the evidence and does not remove the account-level consequence.
2. Appeal through the platform's mechanism — Account Status on Meta, the post's appeal flow elsewhere — and say plainly that the content is authentic, not generated.
3. Attach the strongest evidence you have. In descending order: a C2PA manifest from a camera that supports it, the original file with its embedded metadata and an intact pixel watermark, the collaborator's statement and consent record, the raw capture file.
4. Escalate if the appeal fails. For a business account, the account representative route is faster than the in-app loop.
5. Log it as an incident in the register from 11.4, with dates and outcomes. A pattern of false positives across accounts is a platform problem worth documenting and, where the account is large enough, worth raising publicly.

The preparation that makes all of this fast is unglamorous: keep originals, keep a checksum, and know which of your collaborators can supply a statement within a day.

The design consequence

Put the three mechanisms together and a rule falls out that shapes your visual direction.

Detection is probabilistic, marks are strippable, and a label that lands on authentic content stays while you appeal. In that environment, photorealism is a liability you are choosing. The more your content reads as an authentic record of a real person, the more often classifiers will engage with it, and the more you depend on a review team to agree with you.

Three legitimate responses, none of which is "make it look worse":

• Keep a recognizable visual register. An illustrated, stylized, or otherwise clearly authored look sits outside the realism trigger on TikTok and YouTube, as chapter 11.2 describes, and reduces accidental classifier engagement.
• Declare first and declare always. A declared label is a fact about your account; an undeclared one that a classifier imposes is an accusation, and it arrives on the platform's schedule.
• Keep provenance records regardless of style. The record costs minutes per asset and is the only thing that answers a challenge from any direction.

Where this goes wrong

Treating C2PA metadata as durable. It survives until the first re-encode. Plan for the second publication, not the first.

Assuming a marked file is protected. The Act requires marking and does not penalize removal.

Watermarking without testing survival. A watermark that your platform strips on upload is a cost with no benefit. Run the test.

Re-encoding the master. Every editor pass degrades pixel marks. Archive the generator's original.

Deleting a mislabeled post. You lose the evidence and the label decision stands.

Choosing photorealism without accounting for the classifier. The realism tier costs reach when it goes wrong, and it goes wrong probabilistically.

The gate

1. Which of your assets are labeled, where, and from which mechanism — declaration, classifier, or manifest?
2. Have you run a survival test on each platform you publish to, and what did it show?
3. Where is the provenance record, and does it hold the original file plus a checksum?
4. What is your evidence stack for an appeal, and how fast can you assemble it?
5. Which of your formats trigger the realism test, and is that a choice you made on purpose?

What you leave with

A realistic picture of detection — probabilistic classifiers, strippable manifests, pixel marks that work until they do not — the provenance record that is actually under your control, an appeal procedure for the day a classifier is wrong, and the design rule that photorealism is a compliance decision as much as an aesthetic one.