Blog · Review Essay · Last reviewed June 19, 2026

AI Snake Oil and the Belief Machine of Prediction

Arvind Narayanan and Sayash Kapoor's AI Snake Oil is a book about claim hygiene: do not ask whether something is "AI" in the abstract; ask what kind of system it is, what claim is being made for it, what evidence supports that claim, and who bears the cost when the claim fails.

For this review, AI snake oil means an authority claim wearing an AI label without evidence strong enough for the task, deployment context, affected population, and stakes. The problem is not only bad models. It is the institutional wish to convert uncertainty into obedience.

The Book

AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference was published by Princeton University Press in September 2024. Google Books lists a Princeton University Press edition at 360 pages with ISBN 9780691249643; Amazon's hardback listing gives September 24, 2024 as the publication date, 360 pages, ISBN-10 069124913X, and ISBN-13 9780691249131. Princeton's Center for Information Technology Policy identifies the authors as CITP Director and computer science professor Arvind Narayanan and computer science graduate student Sayash Kapoor.

The book grew out of a public project aimed at separating hype, misinformation, misunderstanding, and real technical progress. Princeton's interview with the authors gives the compact definition: AI snake oil is AI that does not work as advertised and probably cannot work as advertised. This page uses that definition narrowly. A weak claim about an AI hiring product is not evidence that all AI is fake; a good claim about image generation is not evidence that a predictive product can foresee job performance, recidivism, illness, or student success.

The book's structure matters. It distinguishes predictive AI, generative AI, content moderation, social-media recommendation claims, existential-risk arguments, and the social conditions under which AI myths spread. That taxonomy is not pedantry. It is the core of the argument. Many bad AI decisions begin when unlike systems are fused into one glamorous word.

The authors' later "AI as normal technology" essay makes the frame more explicit. AI can be powerful, consequential, and disruptive without being treated as a separate species or as destiny. That point fits the book's practical virtue: it lowers the metaphysical temperature so evidence, deployment context, and institutional incentives can be inspected.

The Category Error

AI Snake Oil is strongest when it treats "AI" as a dangerous umbrella term. Generative chatbots, facial-recognition systems, hiring-screening products, credit scoring, fraud detection, content moderation, recommender systems, scientific prediction, and speculative claims about future agents do not fail or succeed for the same reasons. They should not inherit one another's aura.

This matters because a breakthrough in one area can become marketing fuel for a much weaker claim somewhere else. A fluent language model improves, and suddenly a vendor claims to infer personality from video. Image generation advances, and a school district buys detection software that cannot bear the institutional burden placed on it. Model benchmarks rise, and a workplace dashboard presents prediction as neutral knowledge about workers.

The book's basic hygiene is therefore conceptual. Before evaluating an AI claim, name the task. Is the system generating plausible text, classifying a pattern, ranking applicants, predicting future conduct, moderating speech, summarizing evidence, or recommending action? What is the base rate? What is the ground truth? What would count as success? Who can contest the output? What happens after the output enters a workflow? Without those questions, "AI" becomes a costume worn by any product that wants borrowed authority.

The most useful test is not whether the product uses machine learning. It is whether the evidence matches the exact claim. A model can help transcribe audio and still be unfit to diagnose emotion. A model can summarize a policy and still be unfit to apply it to a benefits denial. A model can generate code and still be unfit to act across production systems without observability, rollback, and accountable review.

Prediction as Institutional Fantasy

The book's most important target is predictive AI in high-stakes settings. Princeton's interview with the authors emphasizes their distinction between generative AI, which dominates public discussion, and predictive AI, which is often already deployed in consequential domains. The examples named there include decisions around jobs, bail, medical care, insurance, and repeated rejection by common vendors.

That focus gives the book its political force. Predictive AI is attractive because institutions want foresight without responsibility. A hiring department wants to know who will succeed before training anyone. A court wants risk without uncertainty. An insurer wants care duration without clinical judgment. A platform wants moderation at scale without a correspondingly scaled public obligation. Prediction promises to turn a contested social future into an administrable present.

The danger is not only inaccuracy. Inaccuracy matters, but the deeper danger is that uncertain inferences become operational facts. A score can deny a job, block a benefit, alter a sentence, trigger surveillance, raise a price, rank a worker, or justify a refusal. Once embedded in workflow, the prediction no longer feels like a claim about the future. It becomes part of the environment in which the affected person must live.

The technical failure modes are familiar: bad labels, proxy variables, small or shifted samples, leakage between training and test data, spurious correlations, changing behavior after deployment, and evaluation against the wrong baseline. Kapoor and Narayanan's separate work on leakage in machine-learning-based science shows why impressive predictive performance can be fragile even in research settings where methods are public enough to examine. In commercial and government deployments, the same problem is often harder to see because the model, data, contract, and workflow are hidden.

The institutional failure mode is just as important. A model can be only mildly predictive and still become powerful if the organization treats its output as permission to stop listening. The person classified by the system then faces two burdens at once: the system's error and the institution's refusal to recognize the score as contestable.

Why the Myths Persist

AI Snake Oil is also a book about belief formation. It asks why weak claims keep circulating even when evidence is thin. The answer is not simply ignorance. AI hype survives because it serves institutions: vendors sell certainty, managers buy legitimacy, journalists get a story, investors get a frontier, policymakers get urgency, and anxious publics get a narrative that makes scattered changes feel like one coherent event.

This is why debunking alone is not enough. The myth has infrastructure. It travels through demos, benchmarks, procurement decks, founder interviews, model-release rituals, trade conferences, analyst reports, screenshots of surprising outputs, and job titles that reward being early more than being accurate. It also travels through fear. A claim that a system may change everything can suspend ordinary evaluation, because delay starts to feel like negligence.

The belief loop is easy to miss. A vendor overclaims. A buyer pilots the product because competitors might. A press story treats adoption as validation. A regulator or standards body responds to the category. The compliance market appears. The product class then looks more real because institutions are now organized around it. The system may still fail at the original task, but the social fact of adoption has made the claim harder to unwind.

Narayanan and Kapoor's corrective is not anti-AI. It is anti-enchantment. Some systems work. Some are useful in limited domains. Some harms are real now. Some fears are displaced from mundane deployment into theatrical futures. The book's discipline is to keep those categories apart long enough for public judgment to function.

Governance and Safety

The governance lesson is straightforward: every AI claim should be tied to an evidence burden. The Federal Trade Commission's Operation AI Comply, announced in September 2024, treated deceptive AI claims and AI-enabled consumer deception as ordinary enforcement problems, not as exceptions created by new technology. The FTC, DOJ, CFPB, and EEOC had already issued a joint statement in 2023 saying they would use existing authorities against discrimination and bias in automated systems.

That matters for the book because "snake oil" is not just a rhetorical insult. It can be a consumer-protection, civil-rights, procurement, labor, education, health, finance, or public-administration problem. A vendor that claims to predict worker productivity, legal outcomes, student risk, medical need, creditworthiness, or criminal behavior should be able to show dated validation evidence, a baseline comparison, subgroup performance, known limits, monitoring, and a path for affected people to challenge the result.

NIST's AI Risk Management Framework gives this demand an operational vocabulary: govern, map, measure, and manage. In snake-oil terms, govern asks who owns the claim; map asks what decision and population the system affects; measure asks whether the evidence actually tests the claim; manage asks what changes when the evidence fails. A benchmark score is not enough if it does not match the deployment context.

The EU AI Act makes the same problem legal for many high-risk AI systems. Its official text regulates systems used in areas such as employment, education, access to essential private and public services, law enforcement, migration, and administration of justice, with obligations around risk management, data governance, technical documentation, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity, post-market monitoring, and conformity assessment. Those duties do not prove any system is safe. They show the kinds of records a serious claim must survive.

For U.S. federal agencies, OMB Memorandum M-25-21 requires risk-management practices for high-impact AI, including pre-deployment testing and AI impact assessments. Read beside AI Snake Oil, the practical standard is not "use AI carefully" in the abstract. It is: define the system, define the claim, test against the real decision, preserve evidence, notify affected people where appropriate, support appeal or remedy, monitor after deployment, and stop use when the claim cannot be substantiated.

The AI-Age Reading

The surrounding review shelf contains books about opaque scoring, digital poorhouses, algorithmic bias, surveillance capitalism, labor platforms, dashboards, bureaucratic metrics, and media systems that train public perception. AI Snake Oil belongs beside them because it supplies a field test for the current moment: what exactly is being claimed, and what evidence would make that claim responsible?

That test is especially useful for agentic AI. As systems move from answer generation toward tool use, procurement, scheduling, coding, writing, customer service, education, medical documentation, and administrative triage, the temptation will be to treat observed fluency as transferable competence. The book warns against that move. A model that can summarize a policy does not automatically deserve authority to apply it. A chatbot that can sound empathetic does not automatically become care. An agent that can complete a web task does not automatically become trustworthy delegation.

The same test applies to AI governance. Regulations, audits, model cards, benchmarks, safety cases, and procurement checklists can become snake oil too if they certify the wrong claim. A disclosure that says a system is AI-generated does not prove truth. A benchmark score does not prove fit for deployment. A fairness metric does not prove justice. A human-in-the-loop label does not prove meaningful human control. A certificate for an organizational process does not prove that a specific product is safe for a specific use.

The agentic version of snake oil is especially subtle. It does not always say "this system can predict the future." It says the system can take care of the task. The responsible response is the same: name the task boundary, tools, credentials, data access, action logs, approval gates, rollback path, incident process, and person accountable for the outcome. If those cannot be named, the product is selling delegation without control.

Where the Book Needs Care

The book's clarity is also its risk. A sharp distinction between workable AI and snake oil can make the reader expect cleaner boundaries than real deployments provide. Many systems are mixed: useful for some people, harmful for others, adequate in low-stakes settings, dangerous when scaled, impressive in a demo, brittle under adversarial pressure, or reliable only when surrounded by human work that marketing ignores.

The authors know this, but readers should resist turning "snake oil" into a universal insult. The category is most useful when tied to a specific claim. "This product cannot predict employee success from facial movements" is stronger than "AI hiring is fake." "This model card does not establish clinical safety" is stronger than "AI medicine is hype." Precision is the difference between critique and counter-hype.

The book also pushes against some superintelligence-centered narratives. That will be valuable for many readers, but it should not become an excuse to ignore frontier-model governance, model misuse, security, concentration of compute, labor displacement, environmental costs, or delegated agency. The better reading is not that only near-term harms matter. It is that every claimed harm, near or far, should be disciplined by mechanisms, evidence, incentives, and deployment context.

There is also a political limit to debunking. A product can fail as science and still succeed as management. If it gives a firm a reason to discipline workers, an agency a reason to deny benefits, a school a reason to surveil students, or a platform a reason to scale moderation cheaply, its institutional value may survive its empirical weakness. The answer is not only better education. It is enforceable rights, procurement discipline, audit access, worker voice, public records, and liability for unsupported claims.

What This Changes

The recurring pattern is simple: an institution wants the world to become legible enough to act on, and a technical system offers a legibility that feels cleaner than the world itself. Predictive AI intensifies that pattern because it makes the future look administrable. It turns a person into a likelihood, a likelihood into a decision, and a decision into a record that later systems may treat as evidence.

AI Snake Oil gives a practical antidote to that loop. Ask what the system is actually doing. Ask whether the target can be predicted. Ask who supplied the labels. Ask whether the metric is the mission. Ask whether a person can appeal. Ask whether the institution is using the model to learn, or using the model to stop listening.

The book's deepest value is that it lowers the temperature without lowering the stakes. AI does not need to be a god, demon, oracle, or species successor to matter. It is enough that institutions are buying systems that classify people, allocate attention, reshape work, mediate knowledge, and convert uncertain claims into operational reality. The cure for snake oil is not cynicism. It is evidence joined to institutional accountability.

Source Discipline

This review separates five source types. Google Books, Amazon, Princeton University, and CITP support book metadata and author context. The authors' exercises, Princeton interview, and Knight First Amendment Institute essay support the book's conceptual frame and later "normal technology" extension. The leakage paper supports the claim that predictive performance can be inflated by methodological errors. FTC, NIST, EUR-Lex, and OMB sources support current governance context. Internal links supply site vocabulary, not independent proof.

The review does not rely on isolated model anecdotes as universal evidence. A single weird chatbot output, impressive benchmark, failed product, or enforcement action cannot prove the nature of AI as a whole. Each claim has to be scoped by system type, task, population, evidence date, deployment setting, and consequence.

This page makes no claim that any AI system is conscious, divine, or AGI. It treats AI systems as engineered tools, products, workflows, interfaces, and institutional decision aids whose authority must be bounded by evidence and contestability.

Sources

Book links are paid affiliate links. As an Amazon Associate I earn from qualifying purchases.

Buy on Amazon Browse Books

Return to Blog · Return to Books