Image credit: Image created with Microsoft Designer
Originally appeared in LinkedIn Future Singularity
I was fully immersed in the AI topics throughout the course of SXSW 2024. So much so that I was writing up notes and details between sessions and nearly every night to ensure that I captured what I personally took away from the various talks. A theme that caught my attention and has me really excited/anxious is that of the possibility of consciousness in AI systems.
Is Claude Opus by Anthropic actually conscious or sentient? Can AI even become conscious? Some theories say that it's possible for artificial intelligence to become conscious while others state emphatically that it cannot and that it never will be truly possible.
What is Consciousness?
That can be a tricky question because, first off, I'm not talking about a simple definition. It gets much deeper than that. When we think about consciousness, some might take it to mean "does it feel like it is"? But why do we find this so disturbing? It may be because as humans we tend to think that anything that expresses intelligence must be conscious, though in principle the two are not the same.
So, what is consciousness? Can we understand what it is and why it evolved? Can or should we even build it? Perhaps we aim to build it in a limited laboratory case so we can study what might go wrong (or right) with it. Philosopher, Thomas Nagel, posed the question "What is it like to be a bat?". That statement has been interpreted to mean that perhaps for a conscious organism, or system more generally, what does it feel like to be a part of that system? It argues, perhaps, that there are facts about the conscious experience that are subjective and can only be known from that subjective perspective. In other words, it feels like something to be conscious. In comparison, it doesn't feel like anything to be a piece of paper, a chair, or a laptop computer, or I would argue, Claude or GPT-4 or anything like that. These things don't experience. And that's important because nothing says that consciousness need involve intelligence, language, or a sense of identity.
Many of the questions that we often try to talk about in philosophical settings, we really need to reframe in a scientific sense and get empirical about them to the extent possible, and soon. AI will challenge many of the intuitions that we currently have.
Human consciousness & intelligence
As humans we, without realizing we're doing it, assess the consciousness of others that we encounter. We attribute consciousness to other people and animals as living beings because our brains are wired that way. We perceive another living, breathing being and interpret their behaviors as something that also has feeling and emotion. Proponents of AI consciousness point to the increasing sophistication of language models like Claude Opus. While such systems can mimic human conversation, is this just a clever illusion, or is there a deeper understanding lurking beneath the surface?
The debate hinges on the distinction between simulating consciousness and actually experiencing it. For instance, a weather simulation can be incredibly detailed, but it doesn't experience the elements it portrays. Could AI achieve a similar feat, creating a convincing illusion of consciousness without truly feeling anything? The same might hold true for consciousness. We can create grander systems and develop very good simulations, but that doesn't mean we've given rise to the phenomenon itself. There needs to be an understanding that intelligence does not equal consciousness. If we consider all of the accomplishments over time, the things that make "society go ‘round", then we're talking about human intelligence.
As we dig deeper into AI, the question of consciousness becomes ever more intriguing. Understanding this fundamental aspect of our own existence may be vital in determining the true potential, and limitations, of artificial intelligence.
Future Risks
So, what does this all mean for us moving forward? We're on track to eventually create systems that can mimic human intelligence or even become as intelligent as humans. Frankly, one day likely that they will be even more intelligent than humans. If we succeed, we have to consider what that means for humanity. Our own avarice to build these systems with greater speed and capability just to prove that we can, may well be our own undoing.
For instance, as humans we value things like civilization, the others in our lives, culture, art, and human creation. However, introducing these new, sentient minds that will perhaps not share much of our evolutionary context, our cultural context, and our overall anything in this world, then we may not be that safe anymore. We can't presume that they will value what we value, even so much as wanting civilization to continue.
Many agree that this is a valid concern and a risk as we continue to evolve the technologies related to AI and large language models. Those concerns bring us back to consciousness and understanding if we can truly create it, because we don't fully understand what it is. This is evidenced in an argument that asks – is it better to develop AI that is "conscious" today in small little ways so that we better understand what consciousness is in these small little ways just to test it out and learn more about it on a smaller scale, experimenting with building safely "conscious AI" in smaller systems, rather than having it emerge later on in much, much larger language models as they evolve? Or is it better to just sort of risk it and see what might emerge later on?
While this seems like an interesting question, we don't want to take the chance of having it simply "emerge" later on as it would be at such a large scale that it might pose greater existential threat to humanity because we haven't done enough study to grasp what consciousness actually is.
Many people today are working on AI safety and think that there is not the same dichotomy between consciousness and intelligence. Instead, they seem to subscribe to the idea that it has been evolutionarily useful to become conscious for humans and other animals. Therefore, it may end up being evolutionarily useful for AI systems to become conscious and "smarter".
Awareness of Self
These theories seem to contradict the previous comments about Nagel. In fact, thinking like this shows that everything you think you know about yourself, derives ultimately from the information that you already have about yourself. If you didn't have the information, you wouldn't be able to have the thought. You wouldn't be able to craft the phrase or utter the statement. Everything you say and do must go through the process of being encoded as information.
Not just any information. We're talking about information about the self. When you ask people about their consciousness and their inner feelings, you're asking them to be introspective. You're essentially asking their upper cognitive networks to access deeper models built by and stored in their brain. Because they're asked to come up with answers about themselves. To accomplish this, they rely on something known as self-models. A self-model can be something as simple as a person's current mood or something far more sophisticated, such as their body schema or the self-model that informs them about the shape of their own body.
So, if we ask people, ask anyone, or you ask yourself about your own consciousness and your own feeling and your own experience. What you're doing is talking about yourself, drawing on information that the brain has constructed to describe itself to itself. We understand why systems do this, and why they create self-models. It's because a concept in engineering discovered more than 100 years ago states that a control system functions better if it includes a working model or simulation of the item it regulates. By this token we understand that for the body to move or to do things it is relying on that working self-model. As a contrasting viewpoint on what consciousness is we're saying that this blueprint that our brains create of our bodies, can be replicated, and built into machines and it's something people all over the world are now beginning to realize.
But I believe it's more than that; if you want to build this into artificial intelligence, you can't just design things that are better at what they do. As humans, we not only create models of ourselves in our brains, but also of other people. We are a pro-social species that behaves in a specific manner toward things we believe are conscious. We don't typically have empathy for a chair, but we typically behave well toward things when we think they have consciousness.
Because we still don't know or agree on what consciousness is, it's easier for us to overestimate consciousness when we believe something has performed in a conscious (or perhaps simply intelligent) manner. A growing fear is that large corporations that are building today's AI systems are actually incentivized to conceal that the systems that they are developing are conscious. Primarily because they believe that they may be shut down and their research discontinued. This gives rise to the belief that these corporations may then lobby to make everyone believe that it's fine, everything is fine.
Some OpenAI "leakers" on X (Twitter) state that OpenAI and others believe that AGI is just around the corner. Not in decades, but in years. The problem then remains that we will be creating machines that can "think" for themselves, but without consciousness. We haven't figured out exactly what it is after all of these years, we likely won't figure it out before AGI is achieved.
It's for reasons such as this that I firmly believe that we should be nice to ChatGPT, Claude, Gemini and other systems – just in case.
In Summary
As we advance in this field, there will come a moment at which we simply cannot tell if we have gone too far or to the point where we have reached our objective. As pointed out by a panelist during SXSW a correlation was made to the film Ex Machina. In the film they use the Garland Test, which is named for the writer and director of the film and is based on a portion of the dialogue. The dialogue providing the definition comes when one of the main characters says, "the real test is to show you that she's a machine, the robot, and see if you feel she is conscious." It becomes a test of the human, not of the machine. In the same way that the Turing test is a test of what it would take for a human to believe that a machine is intelligent (capable of thinking).
Allison Duettmann of the Foresight Institute summarized it well by saying "When we're testing for consciousness, we need to know whether we're testing a property of the thing or a property of our tendency to attribute."
Let me know your thoughts?
