In early 2024, an employee at the Hong Kong office of a global engineering firm joined a video call with their chief financial officer and several colleagues, received urgent instructions to transfer funds — and wired out roughly US$25 million. Every other participant on that call was a deepfake. The video was convincing enough to defeat the oldest authentication protocol we have: "I recognize this person."

For most of photography's history, seeing was believing — not because images couldn't lie, but because lying well took money, skill, and time. Generative models have collapsed that cost to nearly zero for images, audio, and video alike. We cannot un-invent this capability, so the practical question becomes an infrastructure question: when anyone can synthesize anything, what technical machinery lets us verify where a piece of media actually came from?

There are two grand strategies, and they are philosophically opposite. Detection tries to classify synthetic content after the fact. Provenance binds verifiable origin information to content up front. This article argues that provenance — supplemented by watermarking — is winning, while pure ML detection is running into an arms-race ceiling it cannot break through. Then we'll get concrete about what engineers should actually build.

The threat landscape, briefly

Three categories of harm dominate, and each stresses a different part of the ecosystem:

Note what these threats share: none of them require breaking cryptography or compromising servers. They exploit the fact that our systems treat pixels, samples, and frames as self-authenticating. That's exactly the assumption provenance infrastructure attacks.

Two strategies: detect the fake, or prove the real

A useful analogy is antivirus scanning versus code signing.

Detection is antivirus: collect examples of malicious output, train a classifier, deploy it at chokepoints. It's reactive, probabilistic, and always one step behind whatever it classifies.

Provenance is code signing: the origin cryptographically signs information about how the artifact was made, and any verifier can check the signature chain. It's proactive and deterministic — either the signature validates or it doesn't.

Why detection alone loses

Detection research is genuine and improving, but it faces structural headwinds:

  1. Generalization gaps. Detectors trained on one generator's artifacts routinely fail on the next generator. Each new model family — and there is a steady stream of open-source ones — resets the arms race, while detector deployment lags generation capabilities by months.
  2. Adversarial evasion. Classifiers that rely on subtle statistical fingerprints can be washed away deliberately: light noise, recompression, or re-rendering an image through a diffusion model strips many detection cues while keeping the image visually intact.
  3. No positive signal. Detection can only ever say "probably synthetic." It cannot say "definitely authentic." As base rates shift, false positives grow more costly — more on that below.
  4. The liar's dividend compounds. Every publicized detector mistake makes it easier for bad actors to wave off authentic recordings as AI. An unreliable detection regime doesn't just miss fakes; it actively erodes trust in real evidence.

None of this says "delete your detectors." Detection remains valuable as a triage layer at scale. But it cannot be the foundation, because its guarantees are statistical and adversarially fragile. Provenance flips the problem: instead of hunting for traces of fakery, you ask whether the file carries a valid cryptographic claim about its origin — and treat absence as unknown rather than guilty.

C2PA: provenance as a signed edit history

The Coalition for Content Provenance and Authenticity (C2PA) was founded in 2021 by Adobe, Arm, the BBC, Intel, Microsoft, and Truepic, and publishes an open technical specification. The consumer-facing layer is called Content Credentials — the "nutrition label" for media. Behind it sits the Content Authenticity Initiative, whose membership spans thousands of companies and nonprofits across hardware, software, journalism, and photography.

What's inside a manifest

At the heart of C2PA is the manifest: a digitally signed data structure embedded in the asset itself. Its components matter for understanding both its power and its limits:

Verification is not gated behind enterprise tooling. Open-source SDKs exist for Rust, Node.js, Python, and Java, plus the c2patool command-line utility, and free web-based verifiers can inspect any file you drag into them. If your backend runs JavaScript, adding manifest validation is a library dependency, not a platform project.

Who has shipped it

Adoption has moved from press releases to products:

This matters because it covers both sides of the authenticity ledger: cameras prove "a real sensor saw this," and generators prove "a model made this." A healthy ecosystem needs both signatures to be meaningful — absence of a camera credential plus presence of a generator credential is exactly the signal you want at verification time.

What a manifest can — and cannot — prove

Be precise here, because overselling provenance is its own failure mode:

Provenance answers "where did this file come from, provably?" It never answers "is what it depicts true?" — that judgment still requires context, sources, and humans.

Invisible watermarking: provenance that survives stripping

C2PA's hard binding lives in metadata, and metadata is fragile: most social platforms historically stripped EXIF and custom boxes on upload, and some still do. Watermarks attack this weakness by hiding the signal inside the content itself, where ordinary processing preserves it better than sidecar data.

Classical schemes vs learned watermarks

Classical watermarking predates deep learning: least-significant-bit tweaks (fragile), spread-spectrum and frequency-domain embedding (more robust, but deterministic). Deterministic schemes have a known failure mode: once the algorithm is public, targeted removal attacks follow, and several published attacks successfully strip specific fixed-scheme watermarks.

Learned watermarks change the game by embedding during generation itself. Google DeepMind's SynthID is the flagship example: instead of post-processing a finished image, the generative model's decoding process weaves the pattern into the output so it remains coherent with the content, and a paired detector model finds it. SynthID variants now cover images, video, music/audio, and text (Google open-sourced its text-watermarking approach in 2024). Meta applies visible and invisible marks to its own generative outputs, and other labs run similar programs.

Text deserves a special note: watermarking text means biasing token selection toward statistically detectable choices. It works, but paraphrase attacks degrade it, and quality trade-offs must be measured — a reminder that all watermarking lives on a robustness–perceptibility frontier: the harder the watermark, the more it risks subtly changing the content; the subtler it is, the less laundering it survives.

Laundering attacks hurt everyone

The attacker's universal move is laundering: screenshot the image, crop it, rotate it, screenshot again, re-encode aggressively, or push it through a diffusion model's image-to-image mode for a fresh render. Print it and photograph it if you're patient.

These operations:

Laundering degrades all layers simultaneously — but asymmetrically. The defender only needs the signal to survive well enough on realistic distribution paths (screenshots and recompression, not laboratory-grade adversarial filtering), and combining a stripped-manifest lookup via soft bindings with a surviving watermark raises the attacker's cost enormously. Defense in depth isn't a slogan here; it's the entire design rationale.

Where each layer fails

C2PA provenance Invisible watermark ML detection
What it establishes Signed origin + edit history Likely generating model/system Probability of synthesis
Guarantee type Cryptographic Statistical Statistical
Survives metadata stripping No (hard binding) — soft binding helps Yes, partially N/A
Survives crops/re-encode/diffusion round-trip No Partially Poorly
False positive risk Very low — but absence ≠ fake Low High, and consequences land on real people
Who can apply it Anyone holding a key; capture devices, editors, generators Only the generator (learned schemes) Anyone, after the fact
Adversarial resistance Tamper-evident by construction Degrades under laundering Actively evadable
Maturity Shipping across camera/platform ecosystem Shipping in major closed models Research-grade, brittle

Three failure modes deserve emphasis:

Metadata stripping is the default. If your pipeline ingests social-media screenshots, expect no manifests. Treat missing credentials as unknown origin — never as proof of fakery. The whole point of the soft-binding work is to convert "stripped" back into "traceable."

Closed-model opacity breeds justified distrust. Proprietary watermark detectors and private classifiers can't be independently audited; users are asked to trust vendor self-attestation. Expect pressure toward open specifications, third-party verification services, and published robustness benchmarks — and be skeptical of any detection API that won't publish error rates.

Detector false positives cause real harm today. Photojournalists' authentic work and stock photographers' images have been flagged as AI-generated, sometimes with contract or payout consequences; research has repeatedly shown text detectors systematically flag non-native English writing. When a probabilistic verdict gets treated as ground truth, innocent people pay the price — while actual disinformants cite those mistakes as cover. This is why detection verdicts must remain advisory, feeding human review, never auto-publishing accusations.

The conclusion is unavoidable: no single layer suffices. Capture-time signing, generation-time watermarking, platform-level preservation, post-hoc detection as triage, and human editorial judgment are complements, not competitors.

Building it in: guidance for engineers

If you operate a platform, a publishing pipeline, or any product that accepts user media, here's a concrete checklist:

  1. Verify manifests server-side; never trust client claims. A browser saying "this upload is AI-generated" is attacker-controlled input. Run C2PA validation on ingest with an SDK or c2patool, against a maintained trust list. Record the structured result — verified signer, assertion summary, validation errors. "Manifest present but invalid" is itself a strong tamper signal worth alerting on.
  2. Preserve metadata end-to-end in your own pipelines. Image optimizers, transcoding farms, and CDNs routinely strip everything after the pixel data. Audit yours: keep JUMBF/XMP boxes through resize and transcode, re-attach manifests after processing, and add a CI test that pushes a credentialed asset through the pipeline and asserts the credentials survive. Being a link that preserves the chain is the cheapest possible contribution to the ecosystem.
  3. Label what you generate. If your product ships generative features, attach Content Credentials at export, apply an invisible watermark where the model supports it, show users an obvious "AI-generated" marker, and log prompt→asset lineage internally so that months later you can answer "did we make this?" with evidence rather than vibes.
  4. Apply friction to high-risk upload flows. Identity verification, KYC selfies, support channels handling payment changes: rate-limit submissions, require liveness checks, prefer provenance-verifiable capture, and never accept a single image or video clip as sufficient identity proof. The Hong Kong case wasn't defeated by better pixels — it needed process friction that deepfakes can't satisfy.
  5. Expose provenance as data in your APIs, not just as UI badges. Return manifest status, signer identity, and assertion summaries as structured fields so downstream consumers can implement their own policies. Keep the logs: provenance records become forensic evidence when something goes wrong.
  6. Use ML detection as triage, not verdict. Set thresholds conservatively, route borderline cases to human review, publish your false-positive posture internally, and give affected users an appeal path.

The policy floor

Regulators are converging on the same architecture builders are adopting:

Read together with C2PA and SynthID-style watermarking, the direction is clear: mark at creation, carry in both metadata and imperceptible signal, preserve through distribution, disclose at consumption. Implementing provenance now puts you ahead of compliance rather than scrambling behind it.

Key takeaways