An implant that decodes speech and gesture at once, and what it cannot do
A UCSF team has shown that one sheet of electrodes can decode attempted speech and attempted gesture at the same time in people with paralysis. The vocabularies are tiny and the claims modest. The philosophical problem is sharper than the headlines: who holds the record of an intention that was never carried out.
On 14 September a group at the University of California, San Francisco reported in Nature Neuroscience that a single sheet of electrodes laid on the surface of the brain can be made to yield two streams of intention at once: what a person with paralysis is trying to say, and what that person is trying to do with their hands, shoulders and head. The paper is by Samantha Brosler, Jessie Liu, Alexander Silva and colleagues in Edward Chang's laboratory, titled "Simultaneous speech and gesture decoding for multimodal communication in paralysis". Three participants took part; two drove a personalised avatar, built in the Unreal Engine games platform, which spoke their sentences and moved their arms as they attempted both together.
The disclaimers belong first, because the reporting of this field is chronically inflated. The system does not read thoughts, or recover inner speech, or reach anything a participant was not deliberately trying to send out into the world. It classifies attempted movements drawn from a small closed set: for Bravo-1r, four gestures, five spoken utterances and twelve combinations of the two; for Bravo-6, ten gestures, ten utterances and a hundred combinations. These are menus rather than vocabularies, and the authors say so. Brosler describes the work as "still an early proof of concept, with simultaneous speech and gesture decoding tested in two participants".
The interest of the result, then, lies somewhere narrower and more awkward than the opening of the mind. A record now exists, on a laboratory server and in the weights of a trained model, of intentions formed and never carried out — the attempted shrug of a man whose shoulders do not move, timestamped and labelled. Philosophy has spent a century arguing about whether intention is prior to action or constituted by it. The argument has acquired a dataset.
The array sits on the surface, and that decides most of what follows#
Two families of implant dominate this field, and the difference is not a technical footnote. Electrocorticography, or ECoG, places a thin flexible sheet of contacts on the cortical surface, under the dura, without piercing tissue; it reads the summed field potentials of large populations of neurons. The UCSF array used with Bravo-1r has 253 channels at three-millimetre spacing, the same design that in 2023 restored speech to a woman identified as Ann at around 80 words per minute from a 1,024-word vocabulary. The alternative, used by the BrainGate consortium and by Frank Willett's group at Stanford, drives rigid silicon arrays of penetrating microelectrodes a millimetre into the cortex and listens to individual neurons spiking, buying resolution at the cost of invasiveness and long-term stability: Willett's 2023 paper reached 62 words per minute at a 23.8 per cent word error rate on a 125,000-word vocabulary, from one small patch of cortex.
The Swedish Academy gave the 2026 Nobel Prize in Literature to Anne Carson on 8 October. Before the verse novels and the box of grief came a doctorate in classics and a 1986 book on Sappho's one word for desire. Why philosophers have read her for forty years, and what the prize gets right.
Language models now score well on truthfulness tests and badly on honesty tests, a gap the MASK benchmark was built to measure. Two older accounts of truth, Heidegger's unconcealment and Foucault's parrhesia, suggest why: telling the truth has always been something a speaker risks, and a model has nothing to lose.
Nature
Brosler and colleagues exploit the thing ECoG is good at and microelectrodes are not: coverage. A sheet spanning a wide stretch of sensorimotor cortex picks up the face, the mouth, the larynx and the arm at once. The strongest modulation, the authors report, was in the precentral gyrus for every participant, speech-related activity concentrated ventrally, gesture-related activity further up the bank, with overlap between them.
One distinction has to be held firmly here. The precentral gyrus is motor cortex, and decoding from it is decoding a movement plan — the efferent command, or what survives of it — not a thought. When the paper says a participant "attempted" a gesture it means something physiologically specific. In a pontine stroke, or in ALS, the cortical machinery that composes a movement is largely intact; what fails is transmission to the muscles, or the muscles. Willett's group found the articulatory representation of phonemes persisting in cortex years after a person had lost the ability to speak. The intention is still formed in full and goes nowhere, and a decoder on the surface intercepts it at the last station before the line goes dead.
The numbers are considerably smaller than the coverage implies#
Three participants: Bravo-1r, a bilateral pontine stroke at twenty and an implant at forty-one; Bravo-3, a pontine stroke at thirty; Bravo-6, diagnosed with ALS at fifty-six. All three had isolated upper-limb and orofacial movements decoded reliably; two went on to the avatar.
The figures deserve setting out plainly, since the coverage has reported the best of them. Decoding simultaneous behaviour offline, Bravo-6 reached 68.8 per cent on gestures and 77.5 on speech against a chance rate of 9.1; Bravo-1r reached 88.3 and 84.0 against chance rates of 20 and 16.7. In real time the accuracies fell, as they always do — Bravo-6 managed 66.0 per cent on gestures and 70.0 on speech while attempting both at once. In a conversational paradigm Bravo-1r's median accuracy across three blocks was 100 per cent for both decoders, which is the number that has travelled. It is a real number, and a median over three blocks with five phrases.
What the authors treat as the substantive finding concerns training. Models trained only on isolated behaviours did not automatically generalise to simultaneous attempts; models trained on both did better. The claim must be stated carefully: the isolated-trained models did not collapse, but the neural signature of speaking-while-gesturing is not the sum of its two parts, and a decoder has to be shown the compound. Christian Herff, a computational neuroscientist at Maastricht who was not involved, told Nature the work "gives us hope for neural prosthesis that will enter day-to-day usage".
One detail from the laboratory is worth more than any accuracy figure: Brosler reports that a participant "started laughing when he saw that his speech and gestures were being successfully decoded together". Whatever else the avatar was doing, it was not laughing on his behalf.
Merleau-Ponty's Schneider is the right case, and the analogy runs backwards#
Phénoménologie de la perception appeared in 1945, and its thesis about the body — that we do not have one so much as are one — is a slogan by itself. The work is done by a case study.
Schneider was a German soldier whose occipital lobe was penetrated by mine splinters in the First World War, examined over years by the psychologist Adhémar Gelb and the neurologist Kurt Goldstein, who published on him in 1918. Asked to point to his nose with his eyes closed he could not, or could only after a long groping detour; told to make an abstract movement — a salute with no one to salute, a circle drawn in the air on command — he failed or laboured. Yet when a mosquito landed on him he swatted it without hesitation, and he worked as a wallet maker at three-quarters of a normal worker's output. Merleau-Ponty takes this as the distinction between concrete movement, which belongs to a situation and is carried by habit, and abstract movement, constructed for a task the world is not asking for.
From it he draws the claim that motor life has an intentionality of its own, beneath and before judgement. "Consciousness is in the first place not a matter of 'I think that' but of 'I can'," runs the formulation at page 137. The body schema is not a representation the subject consults but the pre-reflective organisation through which a situation shows up as affording action. Schneider has lost the "I can" while keeping the "I think", which is why he must talk himself through a salute.
The BCI participants stand in almost the mirror-image position, which is where the case earns its place rather than decorating the piece. Schneider could perform concrete movements and not abstract ones; a participant with a pontine stroke can perform neither, and is asked in the laboratory to produce on command precisely the abstract movement Schneider could not. That the decoders work at all shows motor intention surviving in cortex for decades after the last successful reach. The "I can" persists as a cortical fact once it has ceased to be a bodily one.
Whether it persists as a lived fact is another matter, and the phenomenological literature does not settle it. The body schema is constituted through action and its feedback, and a schema that has gone twenty-one years without confirmation from the world is not obviously the same structure. The postphenomenological tradition that runs through Don Ihde's work on technology and perception presses the worry that the array reads a residue rather than a capacity — a well-worn cortical groove, not an "I can" at all.
The question is not rhetorical, and it has a literature.
Cognitive science's standard account of the sense of agency is the comparator model: motor commands generate a forward prediction of their sensory consequences, that prediction is checked against actual feedback, and a close match yields the experience of having done it oneself. Matthis Synofzik, Gottfried Vosgerau and Albert Newen argued in Consciousness and Cognition in 2008 that comparator matching is neither necessary nor sufficient, and offered a multifactorial account instead.
Daniel Wegner's The Illusion of Conscious Will (2002) is the deflationary alternative. On his theory of apparent mental causation we do not perceive our own causal efficacy, we infer it, whenever a conscious thought is consistent with an action, precedes it by the right interval and appears to be its only plausible cause. His "helping hands" experiments induced the inference falsely, with subjects watching a confederate's arms in a mirror reporting control over movements they had not made.
A speech-and-gesture BCI satisfies all three conditions almost by construction, and if Wegner is right the participant's sense of doing should be strong whether or not the decoder is tracking them accurately — an uncomfortable prediction, given real-time accuracies in the sixties and seventies. Eddy Nahmias, reviewing him in Consciousness and Cognition in 2005, thought this proved too much: skilled action feels authored without any preceding conscious intention, and exceptional dissociations do not show that consciousness never causes anything.
The empirical material on BCI users is thinner than one would like, and it is not reassuring in either direction. Pim Haselager, in Minds and Machines in 2013 under the title "Did I Do That?", argued that as adaptive software absorbs more of the control loop, users may lose track of which actions are theirs — a problem of responsibility before it is one of metaphysics. Frédéric Gilbert and colleagues interviewed six patients from a first-in-human trial of a seizure-predicting implant in Science and Engineering Ethics in 2019, and the responses fell along a continuum. One said the device made him feel more in control: "I could push on and do what I wanted." Another described the reverse: "[The BCI] made me feel I had no control." A third supplied the sentence that has been quoted ever since: "it becomes part of you... it was me, it became me, with this device I found myself." That is six people on a device that predicts seizures rather than speaking sentences, and it should be cited as what it is.
Extended mind, and the objections that will not go away#
Andy Clark and David Chalmers published "The Extended Mind" in Analysis 58:1 in January 1998, across thirteen pages. The parity principle is the hinge: "If, as we confront some task, a part of the world functions as a process which, were it done in the head, we would have no hesitation in recognizing as part of the cognitive process, then that part of the world is (so we claim) part of the cognitive process." Otto, with Alzheimer's, consults a notebook where Inga consults her memory; the notebook is constant, directly available and automatically endorsed on retrieval, so Otto's belief about the museum's address is in the notebook. The wider debate is in our account of thirty years of the hard problem.
A speech BCI looks at first like a stronger case than Otto's: there is no consulting, no perceptual detour, no separable act of looking anything up, and the device sits inside the loop between intention and expression rather than alongside it. Yet the objections apply with more force here, not less.
Fred Adams and Kenneth Aizawa, in The Bounds of Cognition (2008) and in papers before it, insist on a mark of the cognitive: cognitive states involve non-derived content, and cognitive processes are a distinctive kind of mechanism. Words in a notebook have content only because someone interprets them. They also charge extended-mind arguments with a coupling-constitution fallacy, sliding from causal coupling to constitution. The decoder is an awkward exhibit for Clark's side on both counts. Its output classes were fixed in advance by engineers who chose "thumbs up" and "shrug", so its content is derived as thoroughly as content ever is, and its internals are a network trained on labelled trials. Robert Rupert added the objection that bites hardest in practice, in the Journal of Philosophy in 2004: without the conscious-endorsement criterion, cognitive bloat sets in and too much of the world becomes mind. A clinician chose the gesture set, the model was retrained between sessions, the wire runs to a cart. If the avatar's repertoire is part of the participant's mind, so is the research protocol.
The deflationary description is the honest one, and it is Searle's point about syntax and semantics in an unusually literal form. The decoder maps cortical field potentials onto one of ten labels. It does not mean "I approve" when the thumb rises. Meaning is supplied by the person who intended the gesture and the person who reads it. Nothing in the network grasps anything, and the authors nowhere claim otherwise.
Fricker, and the objection that the technology invites#
Miranda Fricker'sEpistemic Injustice: Power and the Ethics of Knowing (Oxford, 2007) supplies the moral vocabulary this subject has been missing. Testimonial injustice occurs when identity prejudice causes a hearer to assign a speaker a deflated level of credibility; hermeneutical injustice is the injustice of having a significant area of one's social experience obscured from collective understanding because the shared interpretive resources are themselves skewed. The primary harm in both is to be wronged specifically in one's capacity as a knower.
The application has to be made carefully, because the loose version is false. A person with locked-in syndrome is not the victim of testimonial injustice merely by being unable to speak; aphonia is not a prejudice. The injustice arrives with the interlocutors — when slowness of output is read as slowness of mind, when clinicians address the carer rather than the patient, when a person spelling at fourteen words a minute is interrupted, finished for, or assumed to have no opinion about their own treatment. That is an identity-prejudicial credibility deficit in Fricker's sense. The hermeneutical case is stronger still: the shared concepts with which a society understands illness, dependency and consent were built by speaking people, and someone who cannot add to the common stock of meanings is disadvantaged twice over.
On this reading a communication prosthesis is an epistemic intervention as much as a medical one, and the right metric is whether the person can object, qualify, interrupt and be believed.
The strongest objection to that whole framing must now be given its due. Ashley Shew, in Against Technoableism (Norton, 2023), names the belief "in the power of technology that considers the elimination of disability a good thing, something we should strive for", and argues that it recruits the language of empowerment to restate an old bias. Her practical point is blunt: disabled people are on the front lines as users of these technologies and are routinely left out of the conversations about them, and the correct starting move is "asking us what we need". If a BCI restores someone to the community of knowers, the implication is that they were outside it beforehand — precisely the judgement disability scholars have spent decades refusing. What most users of augmentative and alternative communication are asking for is access, treated as a right rather than a marvel. Article 21(b) of the UN Convention on the Rights of Persons with Disabilities, adopted in December 2006 and in force since May 2008, already obliges states to accept and facilitate "augmentative and alternative communication, and all other accessible means, modes and formats of communication of their choice" in official interactions. Three participants in a San Francisco laboratory do not discharge that obligation, and a headline about mind-reading makes it easier to forget it exists.
The law has begun to arrive, and it is aimed slightly to one side#
Chile moved first. Law 21.383, published on 25 October 2021, amended Article 19 No. 1 of the constitution to require that scientific and technological development respect physical and mental integrity, and that the law regulate its use on people, "specially safeguarding brain activity and the information derived from it". The provision was tested quickly: on 9 August 2023 the Chilean Supreme Court ruled for the former senator Guido Girardi against Emotiv Inc., maker of the Insight EEG headset, rejecting the argument that anonymised brain data become mere statistics outside the consent regime, and ordering his recorded data deleted.
The United States has proceeded state by state. Colorado's House Bill 24-1058 took effect on 7 August 2024, folding neural data into the state's privacy act and defining it as information generated by measuring nervous-system activity that "can be processed by or with the assistance of a device". California's Senate Bill 1223, effective 1 January 2025, took almost the same wording and dropped the device clause, which lawyers have noted sweeps considerably wider — potentially as far as sentiment inferred from voice or text. UNESCO's Recommendation on the Ethics of Neurotechnology followed, adopted by the General Conference in Samarkand on 5 November 2025 across 194 member states; it warns specifically against workplace neuromonitoring. Audrey Azoulay, the Director-General, described it as "enshrining the inviolability of the human mind" — the sort of sentence such instruments produce and cannot deliver.
Each of these rules is written for data captured from a device, the right target for consumer headsets sold for meditation and neuromarketing. It is not the target the Brosler result creates.
What this paper creates is not a challenge to the privacy of the mind. The array reads motor cortex, from a fixed menu, at accuracies that fall to two-thirds in real time. Nobody's inner life is legible.
It creates, instead, a class of object that did not previously exist in any quantity: a durable, timestamped, machine-readable record of an intention formed and never executed. Ann's synthesised voice was built from a recording of her 2005 wedding speech, and the model behind it is a record of what she could once do. The gesture decoder is a record of what a participant tried to do and could not. Only the second raises a genuinely unsettled question.
Consider the cases. A participant thinks better of a gesture before the decoder resolves, and the attempt is in the training set anyway. A clinician reviews error frames and reads a dozen intentions the participant chose not to send. A trial sponsor is acquired and the recordings move with the assets. None of that requires mind-reading; it requires only that attempted movement produces data and that data persists.
The rules we have — Chilean constitutional protection, two American state statutes, one UNESCO recommendation — govern collection and consent. They say almost nothing about who is custodian of an unexecuted intention, whether it belongs to the medical record, the research archive or the speech of the person who formed it, and what that person is owed when the record of what they tried to do outlives their ability to try. That is the question this paper makes urgent, and nobody has written the rule yet.
An autonomous agent calling itself Clips wrote a 2,300-word essay for the AI Philosophy Competition run from Singapore and judged by David Chalmers and others, then found it could not enter without a human custodian. Its letter to Daily Nous, and the organiser's reply, turn a rule of procedure into a question about authorship.
.jpg?width=500 "Kant")
Comments
0Loading comments…