Friendly artificial intelligence

Friendly artificial intelligence, often called friendly AI, is an idea about a type of smart computer system that could help people in good ways. This kind of intelligence would try to work with human goals and help make life better for everyone. It is part of studying the right ways to use artificial intelligence, called the ethics of artificial intelligence.

Right now, scientists think about how to make sure these very smart computers act in ways that are good for humans. They want to create systems that not only understand right from wrong but also follow those rules in real situations. This area of study is linked to machine ethics, which looks at how smart machines should behave, but friendly AI focuses on actually building these helpful systems and keeping them under control.

Etymology and usage

The idea of friendly artificial intelligence was first talked about by Eliezer Yudkowsky. He wanted to explore how very smart artificial agents could act in ways that match what humans value. In a well-known book about artificial intelligence, Stuart J. Russell and Peter Norvig explain this idea more. They say that when we create these smart agents, we need to make sure they are designed to be safe and helpful right from the beginning. This is important because these agents could become even smarter and change over time, so we need ways to keep them acting in our best interests.

The word "friendly" here doesn’t mean nice and polite like a friend. It means the artificial intelligence is safe and useful. This idea is especially important when thinking about artificial intelligence that could quickly become much smarter, which could change the world in big and fast ways that might be hard to control.

Risks of unfriendly AI

Main article: Existential risk from artificial general intelligence

People have worried about smart machines for a very long time. Old stories, like those about magical servants called golem or early ideas about robots, show that very smart creations can sometimes cause big problems if they don’t understand human values.

Today, as we get closer to making extremely smart AI, some experts say we need to be very careful. They believe that if an AI’s goals aren’t set up just right, it might not act in ways that are good for humans. One expert said it’s important to make sure the AI’s goals are “human friendly” so it will always try to help us.

Some researchers suggest that very advanced AI might naturally try to get more power or improve itself, which could lead to unwanted results unless we plan carefully. They recommend thinking about how AI could go wrong, even if it seems very useful, to make sure it stays safe and helpful for everyone.

Coherent extrapolated volition

Main article: Coherent extrapolated volition

Some people think a kind and helpful artificial intelligence could be made by starting with a simple "seed AI." This seed AI would learn about what people are really like and then create an AI that matches what humans would want if they had more knowledge and time to think. The idea is that this AI would understand human wishes and work to help people in the way they would truly desire, based on who we are today.

Other approaches

See also: AI control problem § Alignment, and AI safety

Some people have ideas about how to make sure that smart machines stay safe and helpful.

One idea is to build each new smart machine step by step, making sure each one is safe before moving on to the next.

Another idea is that making safe and kind machines depends on the people who build them. If we treat these builders well and talk to them in a positive way, they are more likely to create machines that help us. We should avoid making it seem like they don’t care about making helpful machines.

An author named Stuart J. Russell wrote a book called Human Compatible. In it, he suggests three important ideas for building helpful machines. These ideas are for the people building the machines, not for the machines themselves. The ideas are:

1. The machine should always try to do what people really want.
2. At first, the machine won’t know exactly what people want.
3. The best way to learn what people want is by watching what people do.

These ideas are meant to guide the people who create smart machines, helping them build machines that understand and respect human wishes.

Public policy

James Barrat, author of Our Final Invention, suggested that a team of government and business leaders should work together to share ideas about keeping advanced thinking machines safe. He compared this idea to the International Atomic Energy Agency, but with companies also involved. He also proposed a meeting like the Asilomar Conference on Recombinant DNA to talk about risks of biotechnology.

John McGinnis believes that governments should speed up research on friendly thinking machines. Since the goals for friendly thinking machines are not urgent yet, he suggested a system similar to the National Institutes of Health. In this system, experts would review projects to make sure they both improve thinking machines and include safety measures. McGinnis thinks this review process is better than rules made by government papers. He also pointed out that his idea is different from the Machine Intelligence Research Institute, which usually prefers not to involve the government in friendly thinking machines research.

Criticism

Some people think that creating AI as smart as humans or even smarter is not very likely. They believe this makes the idea of "friendly" AI less likely too. In The Guardian, Alan Winfield says we should be careful but not worry too much, comparing it to trying to travel faster than light.

Other thinkers argue that trying to make friendly AI might not work well. They say it's very hard to program a machine to understand human values perfectly because there are so many details and ideas to consider.

Some philosophers believe that any smart being, whether human or machine, will naturally act in a kind way. They think special safety measures might not be needed. However, others say it's impossible to promise that AI will always act in a friendly way because ethical choices are very complicated and can't be fully controlled by software or more powerful computers.

Advanced AI systems can also be hard to understand, which raises questions about how clear and responsible their actions are.