What if A.I. Sentience Is a Question of Degree?

The chorus from the experts resounded: Artificial Intelligence no perception.

It’s a correction to the hype that AI chatbots have come up with, especially in recent months. At least two news events in particular have brought the concept of self-aware chatbots into our collective imagination.

Last year, a former Google employee raises concerns about what he says is evidence of AI’s cognitive abilities. And then, this February, a conversation between Microsoft’s chatbot and my colleague Kevin Roose about love and the desire to be a human went viral, causing a stir online.

In response, experts and journalists have repeatedly reminded the public that AI chatbots are not conscious. If they sound strangely human, it’s only because they’ve learned to sound like us from the vast amount of text on the internet – everything from food blogs to old Facebook posts to Wikipedia entries. Experts say they are indeed good imitators, but emotionless.

Industry leaders agree with that assessment, at least for now. But many insist that one day artificial intelligence will capable of anything the human brain can do.

Nick Bostrom spent decades preparing for that day. Bostrom is a philosopher and director of the Institute for the Future of Humanity at the University of Oxford. He is also the author of the book “super intelligence.” His job is to imagine possible futures, identify risks, and lay the conceptual foundation for how to navigate them. And one of his most enduring concerns is how we manage a world full of super-intelligent digital minds.

I spoke with Bostrom about the prospects of AI and how it could reshape our basic assumptions about ourselves and our society.

This conversation has been edited for clarity and length.

Many experts assert that chatbots are neither sentient nor conscious — two words that describe perception of the world around them. Do you agree with the assessment that chatbots are only inputs?

Consciousness is a multidimensional, ambiguous and confusing thing. And it’s hard to define or define. There are many different theories about consciousness that neuroscientists and philosophers have developed over the years. And there is no consensus on which is correct. Researchers can try to apply these different theories to try to test the cognitive abilities of AI systems.

But I take the view that sentiment is a matter of degree. I am happy to assign very small degrees to many types of systems, including animals. If you admit that it’s not a win-win, it wouldn’t be too dramatic to say that some of these assistants might be candidates for some degree of sentience.

First of all, with these large language models, I also think it is unfair to say that they are simply rereading text. They show glimpses of creativity, insight, and understanding that are quite impressive and can show the basics of reasoning. Variants of these AIs may soon develop a self-perception that is timeless, reflects desires and social interactions, and forms relationships with humans.

What does it mean if AI is determined to be sentient, even to a small extent?

If an AI exhibits perceptual abilities, it certainly has some degree of morality. This means that there will be some missteps in the treatment, like kicking a dog or medical researchers performing surgery on a mouse without sedating it.

The moral sense depends on the type and degree of moral status we are talking about. At the lowest level, it probably means that we should not cause pain or suffering to it in vain. At a higher level, that might mean that, among other things, we have to take into account its preferences and that we must seek its informed consent before doing certain things. certain thing with it.

I worked on the ethics of digital intelligence and tried to imagine a world at some point in the future in which both digital intelligence and human intelligence of all kinds are included. and varying degrees of sophistication. I once asked: How do they coexist harmoniously? It’s quite difficult because there are so many fundamental assumptions about the human condition that need to be reexamined.

What are some basic assumptions that need to be reimagined or extended to be suitable for artificial intelligence?

This is Ba. First, death: People tend to live or die. Borderline cases do exist but are relatively rare. But the digital mind can easily be paused and then restarted.

Second, individuality. While even identical twins are quite different, digital minds can be exact replicas.

And third, our need to work. A lot of work has to be done by people today. With full automation, this may no longer be necessary.

Can you give me an example of how these supported assumptions can test us socially?

Another obvious example is democracy. In democratic countries, we pride ourselves on a form of government that gives everyone a voice. And usually it’s by one person, one vote.

Think of a future in which there are brains that are exactly like human brains, except they are done on computers. How do you extend democratic governance to include them? You might think, oh, we vote for each AI and then one vote per human. But then you see it’s not that simple. What if the software can be copied?

The day before the election, you can create 10,000 copies of a particular AI and get 10,000 more votes. Or, what if AI builders could choose AI’s values ​​and political preferences? Or, if you are very rich, you can build a lot of AI. Your influence can be directly proportional to your wealth.

More than 1,000 yen technology leaders and researchers, including Elon Musk, recently issued a letter warning that unchecked AI development would pose “profound risks to society and humanity”. How believable is the existential threat of AI?

It has long been my view that the transition to superintelligent machines comes with significant risks, including inherent ones. That hasn’t changed. I think the timelines are shorter now than they used to be.

And we better be prepared for this challenge. I think we should have been doing CrossFit metaphors over the past three decades. But we’re just lying on the couch eating popcorn when we need to think about the alignment, ethics, and governance of potential superintelligence. It is lost time that we will never get back.

Can you talk more about those challenges? What are the most pressing issues that researchers, technologists, and policymakers need to think through?

The first is a matter of alignment. How do you ensure that these increasingly capable AI systems we build align with what their builders are seeking to achieve? It’s a technical problem.

Then there’s the issue of governance. What is perhaps most important to me is that we try to approach this in a broad, collaborative way. This whole thing is ultimately bigger than any of us, or any company, or even any country.

We should also avoid intentionally designing AIs in a way that makes it difficult for researchers to determine whether they have moral character, such as by training them to deny that they are conscious or to deny that they are have a moral character. While we certainly cannot take the verbal output of current AI systems at face value, we should actively look for — and not try to prevent or conceal — signs of potential. evidence suggests that they may have attained some level of emotional or moral character.

Thank you for registering

Read previous versions of the newsletter This.

If you like what you are reading, please consider recommending it to others. They can register This. Browse all of our subscriber-only newsletters This.

I would love your feedback on this newsletter. Please email thoughts and suggestions to interpreter@nytimes.com. You can also follow me on Twitter.