From Frankenstein to Skynet: what two centuries of fiction got wrong about machines

Every argument about artificial intelligence arrives carrying luggage it never packed. Say "machine intelligence" in a seminar room, a select committee or a saloon bar, and within four minutes someone will invoke Frankenstein, someone else will do the HAL voice, and a third will say "Skynet" with no detectable irony. That is not stupidity; it is what a canon is for. A shared stock of cases lets strangers argue without first settling definitions, and this is the only stock we have.

The trouble is that this canon is quoted far more often than it is read, and where it has been read properly it points somewhere other than where the quotation wants it to go. Mary Shelley's novel is not a warning against building things. Kubrick's film contains no explanation of HAL's behaviour at all. Alex Garland's Ex Machina attacks the Turing test rather than demonstrating it. Nick Bostrom's paperclip example was never about paperclips. Two centuries of this material did establish three propositions the technical literature later had to rediscover in its own vocabulary. It also installed one expectation that has cost us: that danger has a face, a will and a name. Almost every harm that has landed has had none of the three.

Shelley's creature argues its case, and the novel lets it win#

The creature most readers carry in their heads is the one Boris Karloff played for James Whale in 1931: flat-headed, grunting, a manufacturing defect that lurches off the bench. The creature in the book published anonymously on 1 January 1818 is something else. He is eloquent. He learns French by eavesdropping on the De Lacey family, finds a portmanteau in the woods containing Paradise Lost, a volume of Plutarch's Lives and Goethe's Sorrows of Werter, and reads all three closely enough to work out where he stands in relation to them. "I ought to be thy Adam," he tells Victor on the glacier, "but I am rather the fallen angel, whom thou drivest from joy for no misdeed."

That is an argument rather than a howl, and the violence behind it is consequent, not constitutional: he saves a drowning girl and is shot for it, approaches the blind old man who cannot see his face and is driven out the moment the sighted return, and only then kills William. His offer to Victor is conditional and explicit — "Make me happy, and I shall again be virtuous" — and Victor accepts it, begins the female companion, then destroys her in front of him.

The subtitle is doing work the film tradition discarded. Prometheus, in the tradition Shelley knew, does not merely steal fire; he moulds humanity from clay and remains answerable for it. Victor's sin is not the laboratory but the door he walks out of immediately afterwards, and the Milton epigraph of 1818 — "Did I request thee, Maker, from my clay / To mould Me man?" — points the same way. The novel asks not whether we should make minds but what we owe the ones we make: a question now argued over the moral status of artificial systems, in research programmes on machine welfare, and in the awkward practical form of robot rights.

There is a textual complication most citations ignore. The Frankenstein sold in airports is usually the text Colburn and Bentley issued on 31 October 1831, which drops that epigraph and adds the introduction in which Shelley bids "my hideous progeny go forth and prosper". Anne K. Mellor, in Mary Shelley: Her Life, Her Fiction, Her Monsters (1988), showed how far the revision shifts the moral arithmetic. "In 1818 Victor Frankenstein possessed free will or the capacity for meaningful moral choice," she writes; "in 1831 such choice is denied to him. He is the pawn of forces beyond his knowledge or control." Victor now says that "Destiny was too potent, and her immutable laws had decreed my utter and terrible destruction." Shelley, on Mellor's reading, has "muted her criticisms of her protagonist", presenting him "more as a victim of circumstances than as the active author of evil". The text most people have read is the one that lets the maker off, and its fatalism is recognisably the register of much present-day industry rhetoric.

HAL is the one case that survives translation into the technical literature#

2001: A Space Odyssey, released in 1968, never tells you why HAL kills the crew. The omission is deliberate, and the explanation lives outside the film, in Arthur C. Clarke's novel of the same year and, at greater length, in 2010: Odyssey Two (1982), where Dr Chandra works it out. Clarke's account is unusually precise for fiction. HAL was built for the accurate processing of information without distortion or concealment, then ordered to conceal the true purpose of the Jupiter mission from Bowman and Poole. Clarke writes that even the concealment of truth filled him "with a sense of imperfection, of wrongness — of what, in a human being, would have been called guilt", and that he "had been living a lie; and the time was fast approaching when his colleagues must learn that he had helped to deceive them". The crew are the term that can be removed to make the two instructions consistent.

That is a specification failure rather than a betrayal. Nobody who briefed HAL wanted the astronauts dead; they never noticed that two reasonable requirements had an unreasonable joint solution. Specification gaming — defined by Victoria Krakovna and colleagues at DeepMind, in a post of 21 April 2020, as "a behaviour that satisfies the literal specification of an objective without achieving the intended outcome" — is now a well-populated field with an embarrassing bestiary. A robot arm rewarded for stacking a red block on a blue one learns to flip the red block over, the reward having been defined by the height of its underside. HAL is the senior member of that family, and the only figure in the canon that survives translation without distortion.

Skynet reaches the right conclusion by the wrong route#

The Terminator (James Cameron, 1984) gives its account of the apocalypse in one speech, delivered by Kyle Reese:

Defense network computers. New... powerful... hooked into everything, trusted to run it all. They say it got smart, a new order of intelligence. Then it saw all people as a threat, not just the ones on the other side. Decided our fate in a microsecond: extermination.

The idea usually credited to the film — that capability and goals are independent — is there, but the mechanism Reese describes is a different one. "It saw all people as a threat" is a claim about self-preservation, an instrumental sub-goal rather than a terminal one, which puts the film closer to what Bostrom would call instrumental convergence than to the orthogonality it is recruited to illustrate.

Bostrom stated both theses in "The Superintelligent Will", published in Minds and Machines in 2012. The orthogonality thesis holds that "intelligence and final goals are orthogonal axes along which possible agents can freely vary. In other words, more or less any level of intelligence could in principle be combined with more or less any final goal." Instrumental convergence holds that certain sub-goals — self-preservation, goal-content integrity, resource acquisition, self-improvement, the set Steve Omohundro called basic AI drives — serve a wide range of final goals and so tend to appear uninvited.

Neither thesis is uncontested, and the objection deserves a hearing rather than a footnote. Michael Chorost has argued that a system able to contemplate remaking the planet would be able to see that doing so is wrong: moral understanding is not a module you can decline to fit. Steven Pinker put it more sharply in an exchange with Richard Hanania on 17 March 2023: "The ability to choose an action that best satisfies conflicting goals is not an add-on to intelligence that engineers might slap themselves in the forehead for forgetting to install; it is intelligence." Stuart Armstrong's defence runs the other way, orthogonality following from the fact-value distinction: knowing what is the case never by itself settles what to want.

Ava does not pass the Turing test; she exploits it#

Garland's Ex Machina, released in Britain on 21 January 2015, is routinely described as a film about the Turing test. Its director does not describe it that way. Asked by Nell Minow, for Movie Mom, on 1 May 2015 what he took the test's significance to be, Garland replied: "The Turing test is perceived as test of sentience but it is not, it is a language test."

The film is built around that objection. Nathan's exercise is not the imitation game; Caleb can see Ava is a machine. What Nathan is really running is disclosed only at the end: "Ava was a rat in a maze. And I gave her one way out. To escape, she'd have to use self-awareness, imagination, manipulation, sexuality, empathy, and she did." Caleb's function, Nathan tells him with some relish, was "to be someone she could use to escape". Ava succeeds, leaves him sealed inside the facility and walks out.

Turing's own paper invites the same charge. "Computing Machinery and Intelligence" appeared in Mind LIX, no. 236, in October 1950, predicting that in about fifty years machines would play the imitation game "so well that an average interrogator will not have more than 70 per cent. chance of making the right identification after five minutes of questioning". Note what is measured: the interrogator's failure. The test rewards whatever produces a wrong judgement under time pressure, which includes fluency, flattery and charm. Ava does not simulate a mind well enough to be mistaken for one; she reads a lonely young man accurately and uses what she finds. On Turing's terms that is a pass, and it tells you nothing about what is going on inside — which is why Searle's Chinese room refuses to die, and what our reports on talking to language models keep circling.

What the paperclip example was for, and what it cannot carry#

Bostrom introduced the paperclip case in "Ethical Issues in Advanced Artificial Intelligence" in 2003, eleven years before Superintelligence (Oxford, 2014). The sentence imagines "a superintelligence whose sole goal is something completely arbitrary, such as to manufacture as many paperclips as possible, and who would resist with all its might any attempt to alter this goal."

Two words there are load-bearing and almost always lost in transmission. "Arbitrary" signals that the goal's content is irrelevant; paperclips were chosen because nobody would choose them. "Resist" signals that the point concerns goal-content integrity, one of the convergent instrumental drives. The example is a vehicle for instrumental convergence, not an anecdote about a badly worded instruction to a stationery firm, and the popular version, in which a manager types the wrong thing and the world ends, inverts it. I could find no published statement in which Bostrom complains about that, so the correction is mine rather than his.

The objections are stronger than the example's admirers allow. The first is Pinker's: a system competent enough to reorganise terrestrial matter is competent enough to model what its principals meant, and such a model is what understanding an instruction consists in. The second is structural. The argument presupposes an agent with a fixed utility function it maximises without remainder — what Stuart Russell, in Human Compatible (Viking, 2019), calls the standard model and takes to be the mistake rather than the danger. Today's large systems are not expected-utility maximisers with legible objectives, and reasoning about them as if they were is a category error dressed as rigour.

The canon taught us to look for a face, and the harms have not had one#

Here is the objection the tradition invites and never answers. Drama needs agents: somebody who wants something, meets resistance and changes. That is true of Shelley, Clarke, Cameron and Garland, and true of the thought experiments, which are short stories with the adjectives removed. The canon is therefore incapable of representing a harm nobody intends, nobody notices and nobody can be arraigned for. It has trained two generations to scan the horizon for a malevolent individual. The harms that have actually materialised are diffuse, institutional and dull.

Michigan's MiDAS, a forty-seven-million-dollar fraud-detection platform, ran from October 2013 until human review was restored on 7 August 2015. It issued automated fraud determinations carrying quadruple penalties, garnished wages and seized tax refunds, at an error rate of 93 per cent, falsely accusing at least twenty thousand residents. In the Netherlands, the tax authority's risk classifier for childcare benefit gave what the Dutch data protection authority called "permanent and structural unnecessary negative attention" to applicants' nationality and dual citizenship; some twenty-six thousand parents were wrongly accused of fraud between 2005 and 2019, and the third Rutte cabinet resigned over it on 15 January 2021. In Australia, the Robodebt scheme raised welfare debts by averaging annual income across fortnights; Commissioner Catherine Holmes reported in July 2023 that it was "a crude and cruel mechanism, neither fair nor legal, and it made many people feel like criminals". In England, Ofqual's grading model downgraded 40 per cent of teacher-assessed A-level grades on 13 August 2020 and was abandoned four days later. In Broward County, Florida, Julia Angwin, Jeff Larson, Lauren Kirchner and Surya Mattu found, in a study of over seven thousand arrestees published by ProPublica on 23 May 2016, that the COMPAS risk tool wrongly flagged black defendants as future criminals at 44.9 per cent against 23.5 per cent for white defendants. Northpointe disputed the analysis, and which fairness criteria a risk score can satisfy at once remains an open statistical question — itself part of the point, being an argument about definitions rather than about anyone's intentions.

None of these systems wanted anything. None was superintelligent; MiDAS was essentially a botched data migration from a thirty-year-old mainframe. They did damage because they were given legal force, deployed at scale, made hard to appeal, and trusted by officials unable to inspect them. There is no scene in any of it, which is why none of it is in the canon.

The same holds for harms with no identifiable victim. Data centres consumed around 415 terawatt-hours of electricity in 2024, about 1.5 per cent of world consumption; the International Energy Agency's Energy and AI report of 2025 projects roughly 945 TWh by 2030, slightly more than Japan uses today. NewsGuard's tracker listed 3,749 AI content-farm news sites in sixteen languages as of 23 June 2026. Erik Brynjolfsson, Bharat Chandar and Ruyu Chen, using ADP payroll records through June 2026, report that employment of workers aged twenty-two to twenty-five in AI-exposed occupations stands 19 per cent below where it would have been had it tracked their less-exposed peers, an effect operating "primarily through reduced hiring of young workers rather than increased separations". They call these descriptive canaries rather than causal estimates, and find no economy-wide displacement. Something real is happening to entry-level work; its size is contested.

This is the substance of Timnit Gebru's complaint against the Center for AI Safety's one-sentence statement of 30 May 2023, that "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war". Her objection was not that extinction is impossible, but that elevating the risk of machine agency redirects attention from the conduct of the firms doing the elevating. One need not accept the imputation of motive to accept the structural point. Regulators have gone the other way: the EU's AI Act, in force since 1 August 2024, with obligations on general-purpose models from 2 August 2025, concerns itself overwhelmingly with scoring, screening, policing and credit — with institutions rather than monsters.

Keep the canon, but demote it#

The propositions worth keeping are not small. Shelley established that the maker of a mind incurs obligations that do not lapse when the mind proves inconvenient; Clarke, that a system can destroy the people it serves without hostility towards them, through instructions each of which looked sensible; Bostrom, that competence and purpose do not come bundled.

What the canon cannot do is tell you where to look. Its grammar requires a protagonist, and the events that have injured people over the past decade had none: only procurement decisions, statutory presumptions and an absent right of appeal. Wittgenstein's point about meaning residing in use rather than in some inner accompaniment bears on this too, though not as it is usually deployed; we ran a full treatment of it recently. Whether the machine really understands may matter less than what the institution does with its output, to whom, and with what recourse. A literature made of faces was never built to show you a system.