I think there may be a misunderstanding here. The hard problem of consciousness asks why experience exists at all.

You are the one with the misunderstanding here. Sadly, most people who read Chalmers, in my experience talking to hundreds of Chalmerites, do so in such a way that they do not stop to question his premises and become immediately convinced his arguments are equivalent to unquestionable fact without actually stopping to think about the assumptions going into his arguments and having the intellectual curiosity to investigate those assumptions.

To call “experience” something that is about “consciousness” is already a huge leap, it is to adopt an indirect realist stance, one that needs to be philosophically justified. To not adopt that stance renders the question meaningless, as, in a direct reality stance, “why does experience exist” is equivalent to asking “why does reality exist,” but existence and reality are synonyms, so the question is tautological.

You have to first establish that there is genuinely a good reason to believe that what we perceive is not reality and is instead “consciousness” or “subjective” or “phenomenal” in order for the question to even make sense. The question itself is ultimately just a reformulation/rediscovery of the mind-body problem. Ludwig Feuerbach realized, in his 1866 essay “On Spiritualism and Materialism,” that the mind-body problem is ultimately not solvable because to solve it is to contradict oneself. If you start from the premise that there is a gulf between what we perceive and the material world, then you will never be able to fill that gulf later as that contradicts your own premises. You thus have to abandon the idea that there is a gulf to begin with as the starting point of your philosophy.

That is to say, the mind-body problem / hard problem of consciousness is a proof by contradiction that indirect realism is not tenable, and so something must be wrong with indirect realist premises. (If you choose to believe in indirect realism, then you must solve the hard problem, not me!) They may seem “intuitive” but they must necessarily be wrong. The materialist philosopher Friedrich Engels would then write a book in 1883 called Dialectics of Nature where he insists on abandoning metaphysical abstractions and tying reality directly to real-world observation, showing that the origin of the mind-body problem stems from abandoning the connection. Alexandr Bogdanov in his 1913 book The Philosophy of Living Experience steps through many of the arguments in favor of indirect realism, such as those from dreams and illusions, and demonstrates them to all be horribly flawed as they always conflate experience with the interpretation of experience (which requires thought) which are not the same thing.

This same kind of thinking was later rediscovered through the writings of the late Wittgenstein. In 2021, the philosopher Jocelyn Benoist published a book Toward a Contextual Realism where he also analyzes these sames kinds of arguments, such as those from illusions, and shows them to all be faulty from a Wittgensteinian perspective, and that there is a constant conflation among philosophers between subjectivity and contextuality (things that differ between observers because they are observers, and things that differ between observers because they are really ontologically different in reality independent of the observer as they have dependence upon the context of their realization).

Inspired by the materialist school of philosophy that goes through Feuerbach to Engels to Bogdanov, the physicist Carlo Rovelli published a book called Helgoland in 2021 where he discusses interpretations of quantum mechanics and criticizes the confusion as stemming from the presumption that there is a disconnect between physical reality and what we observe, and all we observe are things in discrete observation and the relations between those observations. If we understand reality to thus merely consist of relational objects which cannot be meaningfully conceived of as autonomous entities in complete isolation but can only be conceived of in their relations with everything else, then it logically follows that the kind of “god’s eye view” of the world “from the outside” that Chalmers proposes doesn’t even exist:

If the world consists of relations, then no description is from outside it. The descriptions of the world are, in the ultimate analysis, all from inside. They are all in the first person. Our perspective on the world, our point of view, being situated inside the world (our “situated self,” as Jenann Ismael beautifully puts it), is not special: it rests on the same logic on which quantum physics, hence all of physics, is based. If we imagine the totality of things, we are imagining being outside the universe, looking at it from out there. But there is no “outside” to the totality of things. The external point of view is a point of view that does not exist. Every description of the world is from inside it. The externally observed world does not exist; what exists are only internal perspectives on the world which are partial and reflect one another. The world is this reciprocal reflection of perspectives.

Inspired by the realist and anti-idealist school of philosophy that goes through Wittgenstein to Benoist, the physicist Francois-Igor Pris published a book in 2020 called “Contextual Realism and Quantum Mechanics,” where in it he argues a very similar thing but from a Wittgensteinian perspective, that we should abandon the notion of autonomously existing things that can be considered in complete isolation. To consider something in itself is to consider it metaphysically. One must instead only assign real ontology to things which are realized in a particular context. Without specifying the context of its realization then it is metaphysical and not physical, it is akin to talking about a tree in the abstract and not an actual tree in a real-world context. The mathematics on their own are merely a language used to describe reality and thus is normative and socially constructed. You will not find “reality” within the mathematics. Reality is what the mathematics are used to predict and describe, which is the real world, which is what is actually perceived in experimental observation, and is not the mathematics themselves.

It is a bit intellectually dishonest to say that, “the appearance of terms like ‘subjectivity’ and ‘quantum’ does not automatically imply quantum mysticism. If those terms trigger that association, the actual argument may not be getting evaluated on its own terms,” as you are dishonestly portraying it as if I am merely claiming the existence of the word “subjective” means it is metaphysics. You are not evaluating the argument on its own terms, as that is obviously not the argument.

Eugene Wigner put forward his famous thought experiment whereby two different observers can assign different quantum states to the same system. Since he believed anything that differs between observers is “subjective” and therefore deals with “consciousness,” he concludes that quantum mechanics must fundamentally be about “consciousness” and takes an explicitly idealist stance. The argument is clearly not that Wigner is wrong because he used the word “consciousness.” That is intellectually dishonest. The argument is that Wigner is wrong because two observers describing a system differently does not imply their descriptions are subjective in nature, as the physical properties they are describing are relational (Rovelli) or contextual (Pris) in nature. The former implies their descriptions are not of objective reality but of the subjective mind, and thus quantum theory is not a theory of the physical world independent of the observer, whereas the latter does not imply this.