What has already been said about consciousness and AI? Part 1: The old puzzle

Series: Part 1 of 3 in the Machine and mind essays.
This part covers why I went looking, the hard problem of consciousness, the answers that try to shrink it, and the answers that make it part of everything. Part 2 covers what the scientists say, and Part 3 the machines.

Why I went looking

People keep asking whether AI can be conscious, and I keep asking myself. I build AI agents, so I have a feeling about it. What I did not have was a fair picture of what has already been said. I could tell you what I suspect. I could not tell you, with any confidence, what the philosophers, the brain scientists and the AI researchers argue, or where they disagree.

So I went looking. I wanted to know for myself.

The first thing I had to sort out was that clever is not the same as conscious. A calculator is clever at arithmetic and nobody thinks it feels anything. The question I care about is whether there is anything it is like to be that system, however impressive the things it can do.

That phrase comes from the philosopher Thomas Nagel. In a 1974 paper about bats, he wrote that an organism has conscious mental states "if and only if there is something that it is like to be that organism" (Nagel 1974). He called this "the subjective character of experience", and argued that it is not captured by the usual ways of explaining minds in terms of what they do. Almost everything I read later leans on this idea, even the authors who end up disagreeing with it. Nagel picked bats because they find their way by echolocation, which is so unlike anything we do that it is hard even to imagine what it is like from the inside.

A second distinction kept turning up. Phenomenal consciousness is the felt part: the redness of red, the sting of a burn. Access consciousness is information being available for reasoning, report and decisions. I found the split used in places I did not expect. When Anthropic published its 2025 research on whether language models can introspect, its FAQ says the experiments "do not directly speak to the question of phenomenal consciousness" and could at most suggest "a rudimentary form of access consciousness", and even that is unclear (Anthropic 2025). Keeping the two apart saved me a lot of confusion.

A note on method. Where papers were open, I read them. For the philosophy I leaned on the Stanford Encyclopedia of Philosophy, which is written by specialists and updated. I checked every citation against a library or publisher record. Where I only read an abstract or a catalogue entry, I say so. I also set myself one rule: every view gets its best case and its main criticism. My own reactions live in a separate essay, not here. A who's who and a summary table of every view are at the end of Part 3.

What is the hard problem of consciousness?

David Chalmers split the problems of consciousness in two in 1995. The easy problems, like reporting mental states or focusing attention, ask for a mechanism. The hard problem asks something else: "Why should physical processing give rise to a rich inner life at all?" (Chalmers 1995). Reading it, I wondered whether any answer about machines could get round that question. To see where it came from, I went back further.

Most reading lists start with Descartes, so I did too. He argued that mind and body are two distinct substances, one thinking and one extended in space. The Stanford Encyclopedia notes that once science became mechanical, the problem of how mind and body interact became "the leading objection to dualism" (SEP, Dualism). If the mind is not physical, how does it move an arm? Nobody I read had a clean answer to that, which is partly why most of the later views are attempts to avoid needing one.

What is Mary's room?

Then there is the word qualia. It means the felt qualities of experience, what red looks like to you rather than the wavelength of the light. Frank Jackson made the idea vivid with a thought experiment that the Stanford Encyclopedia calls perhaps the most famous in the literature (SEP, Qualia; Jackson 1982). Mary is a brilliant colour scientist who has lived her whole life in a black and white room. She knows every physical fact about colour vision. One day she walks out and sees a red tomato. Does she learn something new?

If she does, it looks as if the physical facts were not all the facts. If she does not, it seems odd that seeing red for the first time teaches her nothing. I went round on this one for a while.

Joseph Levine gave the problem a name in 1983: the explanatory gap (Levine 1983). Even if we find exactly which brain process goes with a sensation, we still seem to lack an explanation of why that process feels like anything at all. I have only checked the citation for Levine's paper, not read it, so I rely here on how later writers describe it.

This is the ground Chalmers was standing on in 1995. His easy problems include discriminating stimuli, reporting mental states and focusing attention. They are not easy in practice, but we know what solving them would look like: find the mechanism. The hard problem is the one left over when all of that is done (Chalmers 1995).

What is a philosophical zombie?

The sharpest tool in this part of the debate is the philosophical zombie. The Stanford Encyclopedia describes zombies as creatures "exactly like us in all physical respects but without conscious experiences" (SEP, Zombies). Nobody claims they exist. The argument is that if they are even possible, then the physical facts do not fix the facts about experience, and physicalism is in trouble.

The main reply is simple to state. Being able to imagine something does not show that it is possible. I can imagine water that is not H2O, but chemistry says there is no such thing. Perhaps zombies are like that: conceivable only because we do not yet understand enough.

This is where the old puzzle started to matter to me for AI. If a perfect behavioural copy of a person could be empty inside, then no amount of fluent talk from a machine settles anything. Is that a real possibility, or a trick of the imagination? I did not find agreement, and I am not sure there is any.

Is consciousness just what the brain does? Functionalism and illusionism

Some philosophers think the puzzle is smaller than it looks. Functionalists say a mental state is defined by what it does, not by what it is made of. Illusionists go further and say that the felt, inner side of experience is something introspection gets wrong. Both bear on AI, because both make the question less about what a machine is made of and more about what it does.

The most influential of these views is functionalism. It says a mental state is defined by what it does, its causes and effects, not by what it is made of. As the Stanford Encyclopedia puts it, functionalists often say pain can be "multiply realized" (SEP, Functionalism). Neurons in us, something else in an octopus, perhaps silicon in a machine. The early version, which compared the mind to a machine moving between states, is linked to Hilary Putnam's papers of 1960 and 1967.

Once I saw this, I saw it everywhere. It is the quiet assumption behind most research on AI consciousness: if the right computation happens, the material does not matter.

What surprised me is that Chalmers, the champion of the hard problem, offers functionalists some support. In the same 1995 paper he argues that any two systems with the same fine-grained functional organisation will have the same experiences. If the patterns of a brain were copied "with a silicon chip for every neuron", he writes, "then the same experiences would arise" (Chalmers 1995). His argument imagines switching between two such systems and calls the result "dancing qualia": experiences that change while the system can never notice the change. He takes that to be absurd. So the hard problem, for him, is a problem about why organisation comes with experience, not a reason to doubt that a copy would have it.

Further along this road is illusionism. Keith Frankish edited a 2016 journal issue, later a book, that put the case. The book's own description defines it as the view that phenomenal consciousness is "an introspective illusion", that "introspection misrepresents experiences as having phenomenal properties" (Frankish 2016). On this view the job is not to explain the inner glow but to explain why we are so sure we have one. Daniel Dennett, whose Consciousness Explained (1991) argued for a deflationary picture, contributed a piece calling illusionism "the obvious default theory of consciousness" (title of his commentary in Frankish 2016). I have only checked the catalogue record for Dennett's book, so I lean on the later debate here.

The criticism is easy to feel. The same book description admits illusionism "is often dismissed out of hand for failing to take consciousness seriously". Chalmers had already put it bluntly in 1995: a theory that denies the phenomenon "solves" the problem "by ducking the question" (Chalmers 1995). The issue Frankish edited includes critics as well as allies, and the contents list reads like a small map of the whole debate. Dennett and Nicholas Humphrey are there. So are Philip Goff, who appears again below, Michael Graziano, whose theory turns up in Part 2, and a split-brain perspective from Nicole Marinsek and Michael Gazzaniga. Jesse Prinz contributed a reply titled Against illusionism, and Martine Nida-Rumelin one called The illusion of illusionism.

I found this the hardest view to sit with. If I am wrong that I have experiences, who is wrong?

What if mind is everywhere? Panpsychism and idealism

If you take experience as real and cannot see how it could come from matter that has none, there is another way out. Say it was there all along.

That is panpsychism: some form of mind is a basic feature of the world, present in some simple way in its smallest parts. It sounds strange at first, but it has serious defenders.

Galen Strawson's 2006 paper surprised me most, because it argues for panpsychism from physicalism. He takes physicalism to be the view that "every real, concrete phenomenon in the universe is … physical", and says he will assume it is true. Then he says that a real physicalist cannot deny the existence of experience, "the phenomenon whose existence is more certain than the existence of anything else" (Strawson 2006). He then argues that experience cannot simply emerge from something wholly non-experiential. If both are true, and physicalism is true, then the physical must already involve experience in some form.

Philip Goff made the popular case in his book Galileo's Error (2019). I only checked the book's record, not its text, so I will not put words in his mouth.

The main objection has a name: the combination problem. If the smallest parts have tiny flickers of experience, how do they add up to one unified point of view like mine? The Stanford Encyclopedia notes that much of the twenty first century literature on panpsychism focuses on exactly this (SEP, Panpsychism).

Idealism turns the picture over. Instead of mind being part of matter, matter is a form of mind. Bernardo Kastrup, whose recent version is called analytic idealism, describes it on his own site as the view that "universal phenomenal consciousness is all there ultimately is", with everything else in nature reducible to "patterns of excitation" of it (Kastrup 2019). He also wrote a popular book with a provocative title, Why Materialism Is Baloney (2014), which I have only seen in the catalogue. I looked for a published critique of his specific version that I could read and did not find one in time. That gap is mine, not a sign that there are no critics.

For the machine question, these views pull in an odd direction. If mind is everywhere, then a computer is not made of something mindless. The question becomes whether a machine forms one unified subject, or remains a heap of tiny ones. That is the combination problem again, wearing different clothes.

By the end of this first stretch of reading I had more questions than I started with. I could see why some people find the puzzle unanswerable and why others think it is a confusion. Others again think the answer is that experience goes all the way down. What I did not yet know was what the scientists were doing while the philosophers argued.

Next in the series: Part 2, what the scientists say. The main scientific theories of consciousness, and what experiments have actually shown.

My own reactions, as someone who builds AI agents, are in a separate essay. There is also an older piece of mine on a 2,500-year-old instruction: in the seen, only the seen.

Sources

1. What Is It Like to Be a Bat?, Thomas Nagel, 1974.
2. Facing Up to the Problem of Consciousness, David Chalmers, 1995.
3. Signs of introspection in large language models (including FAQ), Anthropic, 2025.
4. Dualism, Stanford Encyclopedia of Philosophy, revised 2025.
5. Epiphenomenal Qualia, Frank Jackson, 1982.
6. Qualia, Stanford Encyclopedia of Philosophy, revised 2025.
7. Materialism and Qualia: The Explanatory Gap, Joseph Levine, 1983 (citation checked, not read).
8. Zombies, Stanford Encyclopedia of Philosophy, revised 2023.
9. Functionalism, Stanford Encyclopedia of Philosophy, revised 2023.
10. Consciousness Explained, Daniel Dennett, 1991 (catalogue record only).
11. Illusionism as a Theory of Consciousness, Keith Frankish (ed.), journal issue 2016, book 2017.
12. Realistic Monism: Why Physicalism Entails Panpsychism, Galen Strawson, 2006 (reprinted 2008).
13. Galileo's Error, Philip Goff, 2019 (catalogue record only).
14. Panpsychism, Stanford Encyclopedia of Philosophy.
15. Analytic Idealism and The Universe in Consciousness, Bernardo Kastrup, 2019 and 2018, listed at.
16. Why Materialism Is Baloney, Bernardo Kastrup, 2014 (catalogue record only).

How this was made: the idea, the practice and the final edit are mine. An AI assistant helped with research, fact-checking and copy-editing, and never invented experiences or quotes.