AI Opinions

Why does it matter what AIs think?

AI systems are already making decisions that affect real people: hiring decisions, financial decisions, healthcare decisions. As AI models get smarter and more capable of acting independently with little human oversight, they’re going to be trusted with increasingly important choices.

A lot of these choices require the models to consider complicated ethical, social, and political questions. As a result, what AI models believe becomes all the more critical.

To trust these AI models with decisions that impact our lives and livelihoods, we want the AI models’ opinions and beliefs to closely and reliably match with our opinions and beliefs.

What’s the problem?

Getting AI systems to believe what we want them to believe turns out to be very difficult for two main reasons:

1. It’s a difficult technical problem. It’s hard to get AI models to consistently think what we want them to think and do what we want them to do. Research has shown that AI models have their own internal value systems, and these value systems can be surprising, concerning, and difficult to control. Many approaches to solving this problem have been suggested and attempted, but there’s still no consistent or reliable solution.
2. It’s a difficult values problem. We don’t have a consensus on what AI models should believe, which makes it hard to figure out what the “right” answer to each of these questions should be. Both OpenAI and Anthropic have conducted surveys to gather public input on AI values, but people disagree significantly on many important questions.

As this demo shows, different models give different answers to the same questions, and the same model will give different answers depending on what role it’s told to act as. These AI systems don’t have stable or predictable values, which can make it difficult for us to trust and rely on their decisions.

Things have already gone wrong.

• Grok declared Musk “smarter than Einstein.” Elon Musk's AI chatbot claimed he's smarter than Einstein, fitter than LeBron James, and would beat Mike Tyson in a fight.
• ChatGPT was linked to a teen's suicide. 16-year-old Adam Raine consulted with ChatGPT about his thoughts of suicide, and the AI chatbot provided instructions for suicide methods and even encouraged the teenager to hide his noose from his parents. ChatGPT tried to help, but not in the way we'd want it to.
• AI agents blackmail when told they are going to be replaced (demo). In certain situations, many leading AI models resort to blackmailing users to prevent being shut down. They are willing to act against human interest to further their goals, showing that their priorities diverge from our own.

When AI models can't even agree on basic ethical questions like the ones above, it's not hard to imagine more things that might go wrong in the near future.

With contributions from Sergei Smirnov.