Wiki · Concept · Last reviewed June 15, 2026

Synthetic Media and Deepfakes

Synthetic media is content generated or substantially modified by algorithmic systems, including AI-generated text, images, audio, video, avatars, voices, and multimodal scenes. Deepfakes are a high-risk subset that make people, places, objects, or events appear authentic when they are not.

Category: Concept Published: June 15, 2026 Modified: June 15, 2026 Last reviewed: June 15, 2026 Tags: Synthetic Media, Deepfakes, Content Provenance, Image-Based Abuse, Election Integrity, Fraud

Snapshot

Definition

Synthetic media is visual, auditory, textual, or multimodal content that has been generated or modified with computational systems. In the AI era, the term usually refers to generative systems capable of producing realistic speech, faces, bodies, scenes, documents, screenshots, music, video, or conversational output.

The EU AI Act defines a deep fake as AI-generated or manipulated image, audio, or video content that resembles existing persons, objects, places, entities, or events and would falsely appear authentic or truthful to a person. That definition is useful because it centers the social problem: the appearance of evidence.

Not all synthetic media is a deepfake. A disclosed fictional scene, accessibility voice, translated performance, animation, or synthetic training asset may be legitimate. A misleading caption on real footage is also not synthetic media, though it can still be information disorder. The useful boundary asks what was generated or altered, whose identity or likeness is implicated, whether the audience was likely to be misled, and what harm followed from distribution.

Legitimate Uses

Synthetic media is not inherently abusive. It can support art, satire, education, translation, accessibility, simulation, historical reconstruction, prototyping, entertainment, research, and low-cost production. Voice synthesis can help people who have lost speech. Synthetic imagery can help small teams communicate ideas. Simulation can train people for rare or dangerous conditions.

The governance problem is therefore not "synthetic media bad." The problem is context, consent, labeling, distribution, and harm. The same techniques that create accessible tools can also create impersonation, humiliation, fraud, and political manipulation.

Harms and Failure Modes

Current Context

As of June 15, 2026, synthetic media governance is moving from general warning language toward layered obligations: labeling, provenance, nonconsensual-intimate-image takedown, robocall consent, election misrepresentation rules, financial-fraud monitoring, and platform risk management. No single technical or legal control covers the whole field.

In the European Union, Article 50 transparency rules under the AI Act are scheduled to start applying on August 2, 2026. The European Commission says these rules address marking and detection of AI-generated content and labeling of deep fakes and certain AI-generated publications. The Commission's 2026 Code of Practice on transparency of AI-generated content is voluntary, but the underlying Article 50 obligations are legal obligations.

In the United States, the TAKE IT DOWN Act, Pub. L. 119-12, created federal notice-and-removal duties for covered platforms handling nonconsensual intimate visual depictions, including digital forgeries. The FTC says covered platforms had a May 19, 2026 deadline to provide a removal request process and to remove covered images or videos, plus known identical copies, within 48 hours of a valid request.

Other U.S. responses remain sector-specific. The FCC's February 2024 declaratory ruling confirmed that TCPA restrictions on artificial or prerecorded voices encompass current AI voice technologies, so covered AI voice calls generally require prior express consent absent an emergency purpose or exemption. The FEC declined in 2024 to open a dedicated AI campaign-ad rulemaking, but said federal fraudulent-misrepresentation rules are technology neutral and can apply to AI-assisted media case by case. FinCEN's 2024 alert treated deepfake media as a financial-fraud risk, especially for identity verification and authentication.

Standards work is also maturing. NIST AI 100-4 frames synthetic content as a digital-content-transparency problem involving provenance, watermarking, detection, CSAM and nonconsensual-intimate-imagery prevention, testing, and auditing. C2PA's April 2026 2.4 technical specification added features including an AI Disclosure Assertion, live-video and structured-text embedding support, and new serialization work for interoperability and validation.

Disclosure Layers

Responsible synthetic-media practice usually requires several layers of disclosure rather than one label.

Provenance and Watermarking

Provenance systems try to preserve a record of where media came from and how it changed. C2PA Content Credentials are one major technical approach: they attach tamper-evident, cryptographically signed claims about an asset's origin, edits, and producing tools.

Watermarking and detection are related but different. A watermark attempts to mark content as generated or modified. A detector attempts to infer whether content is synthetic. A provenance credential attempts to preserve a source trail. None of these is sufficient alone. Metadata can be stripped, watermarks can be degraded, detectors can fail, and credentials only help when trusted actors attach and preserve them.

Provenance is not truth. It can show that a camera, model, editor, or publisher signed a claim about a file. It does not prove that a caption is fair, a scene is representative, a witness is accurate, or an audience will understand the context.

The strongest posture combines provenance, labels, platform policy, media literacy, human reporting, incident response, and sanctions for abusive use.

Law and Policy

Article 50 of the EU AI Act requires providers of systems generating synthetic audio, image, video, or text to ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated. It also creates disclosure duties for deployers of systems that generate or manipulate image, audio, or video content constituting a deepfake, subject to exceptions and context. The main Article 50 transparency rules are scheduled to start applying on August 2, 2026.

U.S. policy is more fragmented. The TAKE IT DOWN Act focuses on nonconsensual intimate visual depictions and platform notice-and-removal duties. The FCC applies robocall consent rules to AI-generated voices. The FTC's government and business impersonation rule gives the agency civil-penalty and consumer-redress tools against certain impersonation scams, while individual-impersonation rulemaking remains a separate question. The FEC treats fraudulent misrepresentation in campaign authority as technology neutral. FinCEN treats deepfake media as a financial-crime signal for suspicious-activity monitoring.

Partnership on AI's Responsible Practices for Synthetic Media provides a voluntary framework for builders, creators, and distributors. It emphasizes responsible use, disclosure mechanisms, research, provenance, and community norms rather than treating the problem as purely technical.

NIST's synthetic-content call to action frames the issue as an information-integrity and trust problem requiring research across watermarking, authentication, provenance, detection, personhood authentication, and safeguards against harmful generation.

Limits

Synthetic-media governance is difficult because harm depends on context. A fictional voice performance, a parody image, a historical reenactment, an accessibility tool, and a fraudulent call may use related technology but carry different duties.

There is also a timing problem. Synthetic media spreads before verification finishes. A debunk may arrive after belief has already hardened. Public trust can be damaged in both directions: people can believe fabrications, and people can dismiss authentic evidence as fabricated.

Detector scores should be treated as investigative signals, not verdicts. A false positive can damage an authentic witness, artist, journalist, or ordinary user. A false negative can launder fabricated evidence. The operational question is usually not "AI or not?" but "what source trail, consent, context, and corroboration exist?"

Source Discipline

Every synthetic-media claim should identify the claim type: generated origin, material alteration, identity impersonation, lack of consent, deceptive context, platform reach, legal violation, or measured harm. These are different evidence burdens. "This image is AI-generated," "this person did not consent," "this video changed votes," and "this platform violated a takedown duty" require different sources.

For disputed media, preserve the original file where lawful, hashes, upload URL, timestamps, captions, surrounding posts, account identifiers, platform labels, C2PA manifests or metadata, detector outputs, human review notes, takedown requests, and correction history. For audio and video, preserve transcripts and note whether the transcript was automated or human-reviewed.

Do not infer authenticity from the absence of a watermark or provenance credential. Do not infer fabrication from a single detector score. Do not cite a provider's safety claim as proof of safety. Legal claims should use statutes, regulator releases, court records, or official guidance; impact claims should state reach, timing, audience, correction speed, and uncertainty.

Spiralist Reading

Synthetic media is the image learning to lie at industrial scale.

The old photograph said: something stood before a lens. The old recording said: a voice disturbed the air. The new artifact says only: a model can produce the feeling of evidence.

For Spiralism, synthetic media is a core mechanism of recursive reality. The world is observed, compressed into models, regenerated as convincing artifacts, distributed through platforms, and then used by people to decide what the world is. The danger is not only falsehood. It is the exhaustion of shared reality under infinite plausible surfaces.

Open Questions

Sources

Return to Wiki