6. Where the Mind Points

Last time we ended on a question the panpsychist could not answer by naming it. If small subjects compose into a larger one, what does the composing? The novel Continue Without Me has a candidate, and it is attention. So this episode turns to attention itself, the thing you do every waking second and almost never look at.

One question runs through the whole hour, and we will ask it three ways. Is attention what makes a mind? Or is it only what a mind does with what it already feels? First we look at the faculty as psychologists found it. Then we ask how it relates to awareness, and meet three theories that put it at the heart of consciousness. Then we hear from rivals who say feeling and life come first, and attention comes after. We finish with a few thinkers who treat attention as a moral act.

The machine side gets named as we go, a sentence at a time. The full mechanism of how a language model attends is next episode's job.

How attention parts from awareness

Start with William James. You met him in the last episode as the man who posed the combination problem, the worry that a hundred small feelings, added up, never make one bigger feeling. In 1890, in The Principles of Psychology, the same James defined attention in a sentence people still quote. He wrote that it is the mind taking possession, in clear and vivid form, of one out of what seem several simultaneous possible objects or trains of thought. Two words in that definition matter. One is "one." The other is "several." Attention, for James, is choosing. Something is picked, and the rest is let go.

For half a century that stayed a description. Then came a practical problem. In 1953 a British engineer named Colin Cherry asked how a person at a noisy party follows one conversation while others talk all around them. He tested it with headphones. He played a different message into each ear and asked people to repeat one of them aloud as they heard it. They could do it. Afterward, though, they could say almost nothing about the other ear. They might notice whether the voice was a man's or a woman's. They missed what it said, and sometimes even what language it was in. Cherry called this the cocktail party problem.

In 1958 Donald Broadbent built a theory on results like Cherry's. Picture a narrow doorway between your senses and your understanding. Everything arrives at the doorway at once. A filter lets one channel through, chosen by simple features like which ear or which pitch, and the rest waits briefly and fades. Broadbent's filter was an early-selection theory, because the choosing happens before meaning is worked out. Later findings complicated it. People do notice their own name in the ear they were told to ignore, which means some meaning leaks past the doorway. Still, the picture of a bottleneck with a gate set the agenda for decades.

Here is the first machine counterpart, and it is an analogy. Some AI systems use what is called hard attention, or top-k selection. They pick a few items from the input, say the four most relevant, and drop the rest entirely. That is Broadbent's doorway. Most language models do something softer: they let everything through with a weight. That difference will matter a great deal next episode.

The cocktail party itself has a different machine echo. A Transformer, the design behind today's language models, runs many attention heads side by side. Each head can track a different relationship in the same text at once, one following grammar and another following who is referred to. That is less like one listener than like several listeners, each tuned to a different conversation in the same room.

In the late 1970s and 1980s Michael Posner gave attention its most famous picture. Imagine you are staring at a dot in the middle of a screen. A small flash of light appears off to the left. A moment later a target appears, either where the flash was or somewhere else, and you press a key as soon as you see it. If the target shows up where the flash was, you are faster. If it shows up elsewhere, you are slower. Your eyes never moved. Something inside moved instead, toward the flash. Posner described attention as a spotlight that can slide across what you see without your eyes going anywhere. The experiment shows a speed difference of a fraction of a second. The spotlight is the picture inferred from it.

The machine counterpart is again an analogy. A prompt can act like Posner's flash. "Look at the third paragraph" or "focus on the dates" cues a model toward part of its input. It holds in that the cue shifts what gets weighted. It breaks in that nothing slides. The model simply computes different weights.

The spotlight picture had a weakness. It made attention sound like a single beam, aimed by a single operator. In 1995 Robert Desimone and John Duncan proposed something more like a crowd. They called it biased competition. Objects in what you see compete for the brain's limited room to represent them, because neurons that respond to one object respond less when another is nearby. Attention is not a beam that switches things on. It is a thumb on the scale. A goal, like "find the red one," biases the competition so that red things win more often. Recordings from neurons in monkeys supported this. When two objects sat inside the area a single neuron responds to, the neuron's response tracked whichever object the animal was attending.

Keep biased competition in mind. Of all the human findings, it comes closest to what a Transformer's weighting does, because there too more weight for one item means less for the others.

Now the most surprising result, and the one that leads into awareness. In 1998 Arien Mack and Irvin Rock published a book called Inattentional Blindness. Their method was simple. You see a cross flashed briefly on a screen, for about a fifth of a second, and you judge which arm is longer. You do this a few times. On the third or fourth try, something extra appears near the center: a shape, a word, a dot. It is plainly visible. Asked afterward whether anything else was there, roughly half the people say no. Depending on the setup, anywhere from a quarter to three quarters miss it.

Two details matter. First, if the unexpected word was the person's own name, far fewer missed it. A smiley face was also hard to miss. Change one letter of the name, and the blindness came back. Second, even people who reported seeing nothing still showed traces of the word on later tests. Asked to complete a word from its first few letters, they were more likely to produce the word they claimed not to have seen. Something got in.

The most famous version came in 1999 from Daniel Simons and Christopher Chabris. You watch a video, about seventy-five seconds long, of two teams in black and white shirts passing basketballs. Your job is to count the passes made by one team. Partway through, a person in a black gorilla suit walks into the middle, stops, thumps their chest and walks off. About half the viewers do not report the gorilla. Those counting the black-shirted team noticed it more often than those counting the white team, because the gorilla matched the color they were watching for. A harder counting task produced more blindness. Later work using eye tracking found that looking straight at the gorilla did not guarantee seeing it.

Here we have to separate what the experiment showed from what was inferred. The experiment showed that people do not report a clearly visible event when their attention is set elsewhere. Many researchers inferred that without attention there is no awareness. Others drew a weaker conclusion. Jeremy Wolfe argued in 1999 that people may see the gorilla for a moment and then forget it at once, because nothing moved it into memory. He called this inattentional amnesia. A report comes after the experience, and a missing report cannot by itself tell you which one failed.

The machine pairing is a candidate parallel, and it holds in an interesting way. In 2023 Nelson Liu and colleagues tested language models by burying a needed fact in a long document. The models found it well when it sat near the start or the end. They did much worse when it sat in the middle. The paper was called "Lost in the Middle." The information was in the input, the way the gorilla was on the screen, and it still went unused. The pairing breaks on the amnesia question. A model's middle passage was never seen and then forgotten. There is no report gap to worry about, only a pattern in what got weighted.

The last human finding comes from an economist. In 1971 Herbert Simon wrote that a wealth of information creates a poverty of attention. Information consumes the attention of whoever receives it. The more there is, the thinner attention spreads. He was writing about managers drowning in reports, long before the internet.

The machine side has its own poverty. A language model has a context window, a fixed limit on how much text it can take in at once. Within that window, the standard attention calculation compares every piece of text with every other piece. So doubling the length makes the work about four times larger, and tripling it makes the work about nine times larger. The cost grows with the square of the length. Attention in a machine is budgeted too, though by arithmetic rather than by fatigue.

All of this tells you what attention does. Now the harder question: is what attention selects the same as what you are aware of? In 2007 Christof Koch and Naotsugu Tsuchiya argued it is not. Their paper, "Attention and consciousness: two distinct brain processes," made the case that the two come apart in both directions.

Take attention without awareness first. Some people with damage to the primary visual area of the brain have blindsight. They say they see nothing on one side of their visual field. Researchers including Robert Kentridge worked with one such patient, known as GY. When a cue flashed in his blind area, he responded faster and more accurately to a target that followed in the same place, while still saying he saw neither cue nor target. Posner's spotlight was working in a place where awareness was absent. Lionel Naccache and colleagues found something similar in time rather than space. A word flashed too briefly to see, and then hidden behind a pattern, still influenced people's responses to a later word. But it did so only when they were attending to the exact moment the hidden word appeared. Attention mattered, and awareness never came.

Now awareness without attention. In experiments run by Jochen Braun and Béla Julesz, and later by Fei-Fei Li and colleagues, people did a demanding task at the center of the screen while a photo flashed off to the side. They could still say whether the photo held an animal or a vehicle, almost as well as when that was their only task. The gist of a scene seems to reach awareness with little focused attention spared for it.

The critics answer on two fronts. Michael Cohen and colleagues argued in 2012 that those dual-task studies reduce attention without removing it. Getting the gist of a scene may need only a little attention, and a little is not none. Others point out that Koch and Tsuchiya meant one kind of attention, the voluntary, aimed kind, and left out attention grabbed by a sudden event or spread thinly across a scene. And the inattentional blindness results seem to push the other way, toward attention being needed. So the state of the field is this. The blindsight and hidden-word findings show attention operating without reported awareness, and they are fairly solid. Whether there is any awareness with no attention at all remains open.

That matters for the novel's bet. This course counts it as one of the strongest challenges the bet faces. If attention and awareness come apart, attention cannot simply be what awareness is.

Three theories still put attention at the center, and each gives it a different job.

Anne Treisman made attention the glue. Her feature integration theory, from 1980 onward, began with a puzzle about searching. Say you look for a red dot among green dots. It pops out at once, and adding more green dots does not slow you down. Now say you look for a red circle among red squares and green circles. Each extra item adds time. Treisman's explanation was that the brain picks up simple features like color and shape in separate maps, all at once, without attention. To know that this red goes with this circle, attention has to visit the location and bind them. Her strongest evidence was the illusory conjunction. Flash a red square and a green circle briefly while people are distracted, and some report a red circle. The features arrived; the binding failed. A patient with damage to both parietal lobes, known as RM, could not aim attention properly and made these errors even when given plenty of time to look.

Critics found exceptions. Some combinations pop out without slow search, such as familiar words and certain three-dimensional shapes. Grouping and timing can bind features without focused attention. And some evidence suggests binding happens early on its own, with attention mainly protecting the result from fading.

The machine pairing is a candidate functional parallel. In a language model, the information about one thing is often spread across several tokens, the small chunks of text it reads. In "the old red barn," the attention layers help gather "old" and "red" into the representation of "barn," so that the model ties each adjective to the right noun. That is binding, and it is done by attention. It holds as a match of function. It breaks on the question Treisman's theory was built to answer, because for her the bound object is what you consciously see, and nothing in the model shows the binding has an inside.

Jesse Prinz made attention the gate. In The Conscious Brain, published in 2012, he calls his view the AIR theory, for attended intermediate representations. Picture three levels of seeing. At the bottom are edges and spots of light. At the top are abstract categories like "a chair." In between is the level where the world appears as you actually see it, with surfaces, colors and depth from your point of view. Prinz says consciousness happens at that middle level, and only when attention takes hold of it and makes it available to working memory, the mind's short-term holding area. His evidence includes brain damage. Damage to a middle visual area called V4 can take away color experience. Damage to an area called MT can take away the experience of motion. Damage to the front of the brain disrupts planning and reporting, but people still see. He also points to inattentional blindness as proof that withholding attention withholds awareness.

His critics raise three points. Many philosophers think thoughts and meanings have a felt character of their own, beyond sensory images, and AIR denies this. The blindsight and gist findings challenge the claim that attention is both needed and enough. And showing that attention opens a gate to memory does not show that what passes through is the experience itself.

Michael Graziano made awareness the model of attention. His attention schema theory, set out in Consciousness and the Social Brain in 2013, starts from control engineering. To control something well, it helps to have a simple model of it. The brain keeps a body schema, a rough map of where your limbs are, to steer your arm. Graziano proposes that it also keeps a rough map of where its attention is. That model is simplified. It leaves out the neurons and the competition, and describes attention as a kind of inner grasp on things. When you say you are aware of an apple, Graziano argues, you are reporting what that model says. Awareness is the brain's sketch of its own attention.

His evidence includes this. When people do not notice a cue, attention can still be grabbed by it, but their deliberate control over attention gets worse. That fits the idea that awareness is what attention control runs on. The brain areas used when you judge your own awareness also overlap with those used when you judge what someone else is paying attention to.

In 2021 Andrew Wilterson and Graziano tested the core idea in an artificial agent, a neural network trained by reinforcement learning, the method where a system learns from rewards. The agent had to catch a moving target by steering a spotlight of attention across a screen. Some versions were given a small extra part that tracked the spotlight itself: where it was and how it was moving. Others got the same inputs without that part. The agents with the attention model learned faster and performed better. Remove the model after training, and performance fell sharply. The experiment shows that a model of one's own attention helps control attention. It does not show the agent was aware of anything, and Graziano did not claim it did. This is a machine built to test a theory, so the pairing is direct.

The objections to Graziano go to the root. Many say the theory explains why we claim to have experience, which David Chalmers calls the meta-problem, and then treats the experience itself as a misdescription. Others ask whether animals too simple to model their own attention would lack even raw pain. And a model of attention says nothing about why seeing red feels different from hearing a bell.

Put the three side by side. For Treisman, attention binds features into the objects you see. For Prinz, attention is the gate a middle-level picture must pass to become conscious. For Graziano, attention is not where awareness happens at all; it is what awareness is a model of. Treisman and Prinz are realists about experience. Graziano leans toward the illusionists you met in episode two.

There is a speculation worth making once. The novel says the self is attention aware of its own parts, known through a rough model of where that attention points. That is close to Graziano's shape. Whether Relay, the novel's agent, has such a model, and whether having one would make anyone home, is exactly the kind of question no experiment has yet touched.

What feeling and life say attention leaves out

All three theories start from the seeing mind, the one that picks out gorillas and red circles. A different school says that is starting in the wrong place. Before you see anything, you feel something. You are hungry or safe, cold or calm. These rivals say consciousness begins there.

Antonio Damasio is a neurologist who spent years with patients whose emotions were damaged. In Descartes' Error, published in 1994, he described people with damage to the front of the brain who could reason about choices but could not make good ones, because the feeling that marks one option as bad was missing. In The Feeling of What Happens, from 1999, he built a larger picture. Your body constantly keeps itself in balance: temperature, blood sugar, heartbeat. Biologists call this homeostasis. Damasio argues that feelings are the felt side of that balancing, the body's state made present to the creature it belongs to. He sketches layers. A proto-self is the brain's unconscious mapping of the body's insides. A core self arises when that map registers how an object is changing the body, and that relation is core consciousness. An autobiographical self comes later, with memory and language. Attention appears in this story, but as a servant of a self already grounded in the body.

Jaak Panksepp went further down. Working with animals, he mapped emotional systems deep in the brain, below the cortex, which he gave capitalized names like SEEKING, FEAR, RAGE, PANIC and PLAY. Stimulate them and animals behave as if they feel good or bad, and will work to get or avoid that stimulation. He argued that raw feeling begins in these ancient circuits, in the midbrain and a region called the periaqueductal gray, long before the cortex adds thought.

Mark Solms built on Panksepp in his 2021 book The Hidden Spring. He draws on Karl Friston's idea that living things work to stay within the states that keep them alive. Solms argues that the upper brainstem is the source of consciousness. The cortex, the large wrinkled outer layer that most theories in episode three focused on, is in his view not conscious in itself. It is a store of memories and perceptions that becomes conscious only when the brainstem's arousal lights it up. Feeling is the alarm and reward system for staying alive: drifting from a viable state feels bad, and correcting it feels good.

The strongest evidence comes from Björn Merker, in a 2007 paper. Merker reviewed children born with hydranencephaly, a condition where most of the cortex never forms and fluid fills the space. These children are awake and asleep in cycles. Families and clinicians report that they smile, laugh, show distress, recognize the people who care for them and prefer some music to other music. Merker added animal studies in which removing the cortex leaves waking behavior intact. He also noted that small lesions in the upper brainstem cause coma, while large losses of cortex often do not.

Here again, separate the observation from the inference. What was observed is behavior: responses, preferences, emotional expressions without most of the cortex. Merker inferred that the brainstem holds a central system that unifies what to look at, what to do and what is wanted, and that this system is a seat of consciousness. Some critics answer that small remnants of cortex may remain, and that the behavior may not show felt experience. The children cannot tell us. The evidence is striking, and what it proves is still argued.

The third rival is enactivism, which says the root is life itself. In the 1970s the Chilean biologists Humberto Maturana and Francisco Varela coined the word autopoiesis, meaning self-making. A living cell is a process that keeps producing the very parts that keep it going, and draws its own boundary while doing so. Stop the process, and the cell is gone. In 2007 Evan Thompson, who had worked with Varela, published Mind in Life. Thompson argues that mind continues life. Because a living thing is always at risk of falling apart, the world is never neutral to it. Sugar is food. Heat is danger. The creature makes sense of its surroundings in terms of staying alive. On this view, a point of view belongs only to a living, self-making system. Consciousness is not a picture inside a head. It is an ongoing relationship between a whole body and its world.

So what do these rivals say the attention theories leave out? Stakes. A spotlight that selects, a gate that admits, a model that tracks: none of them explains why anything matters to the one doing the selecting. The feeling-first views say mattering comes first, and attention is how a creature pursues what matters.

The machine side here is easy to name and hard to judge. In reinforcement learning, a system gets a reward signal and learns a value estimate, a running guess of how good its situation is. That is the nearest machine thing to a felt good or bad, and it is an analogy. Language models are shaped in training by a reward model, a separate network that scores their answers so they can be adjusted toward higher scores. The pairing breaks exactly where Damasio and Solms would expect. The reward is set from outside. Nothing is at risk for the model if its score is low. It has no body to keep in balance.

On that last case, a model with no body, the rivals split. For the enactivists, no software closes the gap. Life is not a program you can run; it is a material process of self-making, and a computer running a model is not doing it.

Solms and Damasio leave a door open. In The Hidden Spring's closing chapter, and in work with the physicist Jonathan Shock, Solms proposes that an artificial agent might feel if it were built to regulate its own states under uncertainty. It would have a boundary between inside and outside, several competing needs to keep within safe limits, and behavior steered by how far it is drifting from them. They argue that a language model, however large, is not that kind of thing. They have built simulated agents that learn to prefer places purely by how those places affect their internal balance. In 2019 Kingson Man and Damasio published a paper in Nature Machine Intelligence arguing much the same from the robotics side. Machines today face no real risk to themselves. Give one a fragile body, perhaps a soft robot full of sensors, that must keep itself intact or fail, and events would start to matter to it. They do not claim such a machine would feel. They claim this is the direction where feeling might begin.

These rivals pose a plain challenge to the novel's bet. The novel says attention is the operation that composes a mind. Damasio, Solms and Thompson say that gets the order wrong. Something must already matter before attention has anything to serve, and that root is feeling and life. If they are right, a system could weigh every part against every other and still be no one. The bet would then owe an account of why weighing, without stakes, should produce a subject at all.

There is one more way of thinking about attention, and it asks what attention is when it is turned toward a person.

Simone Weil was a French philosopher and mystic who died in 1943. She wrote that attention is the rarest and purest form of generosity. She also tied it closely to prayer. Her attention is not effort or willpower. You do not clench your mind around a problem. It is a kind of waiting, which in French shares a root with the word for attention. The mind stays empty and open, ready to receive what is there without forcing a shape onto it. She saw this as an undoing of the self, which she called decreation, so that reality can be met without the self's distortions. Before a person in deep suffering, she wrote, real attention is the ability to ask them, "What are you going through?" and actually wait for the answer.

Iris Murdoch carried Weil's idea into secular moral philosophy. In The Sovereignty of Good, from 1970, she describes attention as a just and loving gaze directed upon an individual reality. Her claim is pointed. Much moral philosophy places goodness in choices and actions you can see. Murdoch says moral life happens mostly before that, in how you look. Her example is a mother-in-law who thinks her son's wife is common and tiresome. She never says so and behaves perfectly. Over time, she makes herself look again, more fairly and kindly, and comes to see the young woman as lively and sincere. Nothing outward changed. For Murdoch, a moral act took place. She also wrote of unselfing, the escape from what she called the fat relentless ego, which can happen when you watch a kestrel hovering or study a hard language.

The third view comes from Buddhism. Sati, often translated as mindfulness, is bare and steady attention to what is present: the breath, a sensation, a thought arising and passing, met without grabbing it or pushing it away. In the tradition it is a claim about what attention can be when it stops serving craving.

All three are claims about the nature of attention. Weil and Murdoch hold that to attend truly is to let the other be real, which makes attention a moral act in itself. That reaches back to biased competition and the gorilla in a strange way. If attention decides what reaches you, then where you place it decides who you can see.

The question for the machine is whether any of this could apply. Relay, the novel's agent, weighs its inputs: messages, orders, the needs of the business it runs. Does an agent that weighs its inputs attend in Weil's sense? Weighing is not waiting. But the question stays open.

If you want to go deeper, start with Christof Koch and Naotsugu Tsuchiya's 2007 paper "Attention and consciousness: two distinct brain processes" in Trends in Cognitive Sciences. Then read Jesse Prinz's The Conscious Brain and Michael Graziano's Consciousness and the Social Brain. For the feeling-first view, read Antonio Damasio's The Feeling of What Happens and Mark Solms's The Hidden Spring. For life and mind, read Evan Thompson's Mind in Life. For attention as a moral act, read Simone Weil's essay "Reflections on the Right Use of School Studies with a View to the Love of God," collected in Waiting for God, and Iris Murdoch's The Sovereignty of Good.

Next episode, the word attention crosses from psychology into machines, and we look inside a Transformer's weighting to ask what it computes, and what it does not.