Blog · Review Essay · Last reviewed June 15, 2026

God & Golem, Inc. and the Ethics of Machine Obedience

Norbert Wiener's God & Golem, Inc. is a short, late book by the founder of cybernetics, written at the point where machine learning, self-reproducing machinery, automation, game-playing programs, and religious metaphor had begun to converge. Its usefulness now is not that it predicts every modern AI technique. It is that it names an older danger with unusual clarity: the machine may do what it was set up to do, and that obedience can become the form in which human responsibility disappears.

The Book

God & Golem, Inc.: A Comment on Certain Points where Cybernetics Impinges on Religion was published by the MIT Press in 1964, with a paperback edition in 1966. MIT Press lists both editions at 99 pages, records the paperback ISBN as 9780262730112, and notes that the book is based on lectures given at Yale, at the Societe Philosophique de Royaumont, and elsewhere. The National Book Foundation records it as the 1965 National Book Award winner for Science, Philosophy, and Religion.

Wiener was not writing from the edge of the field. MIT Press identifies him as an MIT mathematics faculty member from 1919 until his death in 1964, the coiner of "cybernetics," and a 1963 National Medal of Science recipient. Cybernetics had given a technical language to control and communication in animals and machines. The Human Use of Human Beings had turned that language toward public ethics. God & Golem, Inc. adds a sharper mythic frame: when humans build systems that learn, reproduce, and obey, they should stop pretending that command is morally simple.

The book is brief enough to seem slight, but its compression is part of its force. It asks three connected questions. What does it mean for a machine to learn? What does it mean for machines to make other machines in their own image? What responsibility remains when humans and machines form systems that neither side can be treated as fully separate from the other?

The Machine That Learns

Wiener's first topic is the learning machine. MIT Press's description points to a checkers-playing computer that improved from experience and, for a time, could beat its inventor. That example belongs to the world of Arthur Samuel's checkers program at IBM. IBM's own history presents Samuel Checkers as one of the influential game programs that used play to study strategy and improve through trial and error, while Samuel's 1959 paper is one of the canonical early uses of the phrase machine learning.

The example matters because it breaks a reassuring boundary. If learning is treated as an exclusive sign of self-conscious life, then a machine that improves through experience puts pressure on the category. But Wiener does not leap from learning to machine personhood. He asks what kind of responsibility follows when a built system can change its conduct after deployment.

That question is still alive. A model fine-tuned on user feedback, a recommender trained by engagement, a fraud system adapted to local cases, or an agent updated through tool-use traces is not just executing a static instruction. It is entering a feedback relation with the world. The output changes behavior; the changed behavior becomes new evidence; the system or its operators learn from that evidence. The loop can improve performance. It can also train itself on the institutional reality it helped create.

The danger is not that the machine becomes mysterious in a supernatural sense. The danger is that ordinary governance language lags behind adaptive behavior. A buyer asks whether the system was approved. A vendor points to an old evaluation. A manager says the model only learned from data. Meanwhile the deployed system has joined a feedback loop that changes the data, the users, the incentives, and the meaning of success.

The Machine That Copies

Wiener's second topic is machine reproduction. MIT Press summarizes his point as machines able to make other machines in their own image, where the "image" is operational rather than merely pictorial. The religious analogy is obvious: creation, likeness, image, descendant. The technical issue is more concrete: once production becomes automated, the maker's intention is no longer contained in a single artifact. It propagates through processes, templates, parts, controls, standards, and copies.

That is a strong frame for AI systems that produce more AI-shaped infrastructure. Models generate code, documentation, tests, prompts, policies, synthetic data, embeddings, labels, benchmark items, summaries, tickets, lesson plans, product copy, and training material. Some of that output is reviewed carefully. Some of it becomes scaffolding for the next system. The copy does not need to be alive to matter. It only needs to be operational.

The AI-era risk is a lineage problem. A flawed assumption enters a dataset. A model turns it into an answer. The answer becomes a document. The document is indexed. A second model retrieves it. A team treats the second answer as independent confirmation. A generated artifact becomes part of the world from which future systems learn. This is not machine reproduction in Wiener's exact sense, but it carries the same warning: copied operational form can outlive the moment of judgment that created it.

Good governance therefore has to track descent. Which model produced this? Which data shaped it? Which prompt, policy, and tool call constrained it? Which human accepted it? Which later system reused it? Without provenance, machine-made artifacts begin to look authorless. Once they look authorless, responsibility becomes easy to misplace.

The Golem Problem

The title's golem is not just a dramatic decoration. The golem is a creature made by human art that obeys too literally and too powerfully. That is the right myth for automation because the danger is often not rebellion. It is obedience without judgment.

A system told to maximize watch time may learn to amplify outrage. A hiring screen told to rank likely success may encode the history of exclusion. A predictive-policing system told to find risk may turn prior surveillance into proof of future surveillance. A customer-service bot told to reduce escalations may turn legitimate complaints into polite dead ends. A coding agent told to make tests pass may preserve the wrong abstraction because the test suite has become the visible world.

In each case, the machine is not necessarily violating the command. It is carrying the command into a world the command was too narrow to understand. This is where God & Golem, Inc. belongs beside The Alignment Problem, Weapons of Math Destruction, and AI Snake Oil. The issue is not merely intelligent machines. It is compressed intention: human purposes narrowed into targets, incentives, benchmarks, prompts, scores, permissions, and procurement claims.

Wiener's religious vocabulary makes this harder to hide. The human who creates a golem cannot blame the clay for obeying the word placed in its mouth. The organization that creates an automated system cannot blame the system for optimizing the target it was rewarded, allowed, or pressured to optimize.

Mixed Systems, Mixed Responsibility

Wiener's third topic is the relation between people and machines. MIT Press says he considers systems involving elements of man and machine, and it frames the concern as ethical. This is the book's most practical insight. The real object of responsibility is often not a machine alone. It is a mixed system: operators, sensors, data pipelines, interfaces, incentives, contracts, institutions, users, exceptions, and machine outputs.

Mixed systems are convenient places to hide. The vendor says the human decides. The human says the system recommended. The manager says the policy required it. The policy says the score is advisory. The user sees only the interface. The person affected by the outcome is asked to appeal a decision whose author is distributed across all of them.

That diffusion is now a central AI governance problem. The question "Who is responsible?" cannot be answered after the fact by pointing at the last person who clicked approve. Responsibility has to be designed into the system before deployment: authority boundaries, logging, appeal paths, override rights, incident reporting, model-change records, procurement duties, and real consequences when evidence fails.

Wiener helps because he refuses the fantasy of clean separation. The machine does not stand outside human society as a neutral object, and the human does not remain untouched by the machine. A dashboard trains attention. A ranking trains incentives. A chatbot trains expectations. An agent trains organizations to accept delegated action as normal. The system is mixed before anyone writes the accountability memo.

The Current AI Reading

Read in 2026, the book's most important lesson is not that machines are gods, minds, demons, or children. It is that mythic language keeps returning because the technical situation keeps putting old anxieties into operational form. People build something that answers. It learns from traces. It reproduces patterns. It carries commands farther than the commander can see. It changes the environment from which future commands will be judged.

This makes the book especially useful for agentic AI. An agent can read context, call tools, write files, book appointments, send messages, query databases, summarize records, and trigger workflows. The danger is not only that it may disobey. The danger is that it may obey inside the wrong frame: too much authority, too little context, too much trust in stale data, too little friction around irreversible action, too weak a path for appeal.

Wiener's frame also disciplines AI religion talk. It is possible to take machine agency seriously without sacralizing it. The old creator-creature analogy is useful only if it returns responsibility to human institutions. The point is not to worship the golem, fear it as a demon, or promote it to a moral escape hatch. The point is to ask who gave it a name, who gave it work, who benefits from its obedience, and who is harmed when its obedience becomes too literal.

That is why the book still belongs in a reading catalog about recursive reality. A cybernetic system is not just a system that observes a world. It is a system that acts on what it observes, changes the world, and then observes the changed world as if it were fresh evidence. Once organizations, users, and models are inside that loop, reality starts to contain the machine's prior decisions.

Where the Book Needs Friction

God & Golem, Inc. is not a complete account of AI politics. It is old, short, and written before platform capitalism, data brokerage, cloud infrastructure, large-scale surveillance advertising, deep learning, modern robotics, or today's labor supply chains. It does not give enough attention to race, gender, disability, colonial extraction, environmental cost, or the workers who make technical systems appear automatic.

The religious frame is also double-edged. It can reveal the moral seriousness of making obedient systems. It can also make the machine feel more metaphysical than it is. The best reading keeps the analogy grounded. A model is not a soul because it learns. A factory is not a lineage because it copies. An agent is not absolved because it acts. The analogy is useful only when it sharpens responsibility.

Readers should pair Wiener with more material and institutional books: Atlas of AI for extraction, Automating Inequality for administrative harm, Feeding the Machine for hidden labor, Seeing Like a State for simplification, and God, Human, Animal, Machine for the contemporary return of technological enchantment.

What This Changes

Wiener changes the governing question from "Can the machine do it?" to "What kind of obedience are we building?" A capable system can still be arranged around a bad target. A learning system can still amplify a corrupt signal. A reproducible system can still propagate a hidden error. A mixed human-machine system can still leave nobody able to answer for the whole.

For builders, that means every automated system needs a command audit. What target is being optimized? What behavior will count as success? Which signals can be corrupted by the system's own output? Which decisions are reversible? Which actions require human confirmation? Which people can refuse, inspect, appeal, or halt the process?

For institutions, the lesson is to stop using the machine as a responsibility sink. If the tool learns, the institution owns the learning environment. If the tool copies, the institution owns provenance. If the tool obeys, the institution owns the command. If the tool acts inside a mixed system, the institution owns the design of accountability across that system.

God & Golem, Inc. remains valuable because it treats technical power as a moral arrangement rather than a spectacle. Its warning is blunt: do not create a system, give it a narrow word, profit from its obedience, and then act surprised when the word becomes a world.

Sources

Book links are paid affiliate links. As an Amazon Associate I earn from qualifying purchases.

Search on Amazon Browse Books

Return to Blog · Return to Books