Date

Author

Денис Куренов, Анна Иванцова

Source

Kholod

Preserved copy

Internet Archive

Translated material

"The future of peaceful coexistence of people is easier to entrust chat bots"

In order not to miss the main materials of the "cold", subscribe to our Instagram and telegram .

Speaking as widely as possible, what would you now identify the basic levels of ethical issues related to the development of AI?

- I would allocate three levels. The first is the level of absolutely necessary, but the least prominent to society - with regard to engineering discipline called Alignment, or “alignment”. Hundreds of experts around the world are working on the creation of metrics and technical solutions for filtering and monitoring AI models before they enter the market. It is important to understand that any AI system available to users is not just a generative model, but the result of labor of a huge number of people engaged in ethics at the applied level.

The second level is the influence of AI on a society that is already noticeable today in many areas. Obviously, this will affect the labor market. There are already disputes about whether we still need people in professions related to writing texts: lawyers, journalists, editors and so on. But the most important thing is the influence on education, on the teaching of the human brain.

AI has become available to all people, and we must understand how to build a learning system so that the human brain, which is a large neural network of biological neurons, can effectively study and develop in conditions of interaction with artificial neuralates. How to build a system of education in a world where generative AI exists? What do you need to teach a person? These are one of the key questions that is facing us today.

The third aspect of the ethics of AI lies in the philosophical plane. We are talking about a rethinking of human anthropology, his place in the world, where not only people who speak natural languages, but also inhuman agents coexist ( they are described below. - approx. “Cold” ). How will this affect the self -determination of man, on his position in the world?

Unlike the second level regarding the social sphere, here we are faced with deep anthropological and philosophical problems. It is precisely for their understanding that it seems to me that the tools that I try to use in my book are needed: comparison with angels, gods and other inhuman persons. Such analogies help to understand who a person becomes when he speaks his own language with an angel or with a car.

How do you feel about various Alarmist views on AI? For example, to the statements of Ilon Mask and Steve Vozniak that the AI systems pose an unimaginable threat to society?

- Such statements are more likely to reflect sociocultural features than real engineering threats. In Protestant societies, especially in the United States, a tendency to binary oppositions is common. We observed this with an example of attitudes towards nanotechnologies and experiments on a large ADRO Collider. Many public figures also perceived them as a source of the Apocalypse.

Such binary thinking is now projected onto public consciousness, forming a distorted idea of AI. And here, oddly enough, AI itself can become an instrument for solving the problem. It seems to me that in the mitigation of the radical views of people, you can count only on language models. I would not rely on the common sense of the masses. Chat bots trained to recognize manifestations of binary thinking and offer alternative points of view should be able to correct the polarization of opinions and call for some more moderate solutions.

Imagine that soon all mobile phones will be equipped with AI systems that will continuously direct users to more balanced judgments. Such noving (incentives that push people in a certain direction, leaving the freedom of choice behind them. - approx. "Cold") can manifest itself in various fields - from advice on a healthy lifestyle to the formation of a tolerant attitude to different points of view.

It turns out such a paradoxical conclusion: it is easier to entrust the future of the peaceful coexistence of people, not people. People, as we see, are not doing this now very effective.

That is, earlier the gods - inhuman agents in the mythological consciousness, were engaged in this “subtle setting” and the names of humanity. And in the future, other inhuman agents-chat bots and AI can in their place?

- The essence remains the same - only the language of the description changes. We switched from talking about the divine punishment to the discussion of a fire-tuning, but inhuman agents affecting a person and society were and remain. Previously, they existed in myths, today - in technological reality. The question is how to interact all to direct the development of society in the desired direction. If in the past cults and rituals were used for this, today they are occupied by “thin-tuning”, “alignment” (alignment) and other methods of control of AI. These are all the same attempts to establish a dialogue with forces exceeding human understanding, only expressed in the language of modern science and technology.

The appearance of ChatGPT at the end of 2022 was so revolutionary and unexpected that the hot disputes around the topic of the latest versions of chat bots are still not subsided. Your committee is at the forefront of the study of AI ethics in the European Union. Did you foresee the appearance of something like that?

- Yes, sure. The appearance of ChatGPT caused excitement in the general public, but experts have long observed the development of this technology. Back in 2021, our committee published an opinion on the ethical issues of chatbots. The next opinion about the ethics of generative AI was ready to exit almost immediately after the ChatGPT launch. My book “Parole de Machines” ( “Word of Machines” is published in French at Humensciences. - Approx. “Cold” ), released a few months later, was also written before ChatGPT release.

We monitored the development of transformers architecture ( type of neural network architecture, which is suitable for processing data sequences. - Approx. “Cold”) and generative networks ( architecture of deeper training. - approx. “Cold”) , and by 2019 it became obvious to what everything was going on. Personally, I began to deal with chat bots ethics seven years ago. In the already mentioned work of our Committee of 2021, there are references to transformers, about future changes. Actually, it says black and white there that the world will no longer be the same. This is what we are observing today.

Transformers and generative networks are special terms. Let's first figure out how chat bots work.

- There are two main approaches. The first is the so -called symbolic AI. This is such a large decision -making tree. For example, if it is raining, you need to take an umbrella. And then there are thousands and thousands of such rules. Even today, Siri is largely based on this approach.

The second approach is neural networks. They are based not on the rules, but consist of artificial neurons, which are a kind of primitive calculator, a very simple computing system. But if you combine millions, billions, hundreds of billions of such neurons, then this huge network begins to demonstrate complex behavior, which cannot be predicted, looking at separate neurons. This is like in statistical physics: gas consists of simple molecules, but in general it behaves very difficult.

People learned to build neural networks back in the 1960s, but only by the end of the 2000s they were able to create truly large ones and began to train them effectively. And then interesting results began to appear.

When did they become a revolution?

- The big revolution, which gave rise to what we call the generative AI today, occurred in 2017. Then a famous article called "Attention is all that is needed." For the first time, all the puzzle pieces were collected in it and the architecture of neural networks called transformer was proposed. It was the transformers that laid the beginning of the entire boom of generative AI.

The first transformers were still quite small, but by 2019 it became clear that their generative opportunities were phenomenal. They generated the text much better than any symbolic systems. Therefore, almost all chat bots in recent years have switched from symbolic AI to transformers.

In a nutshell, a transformer is a neural network that studies on a huge body of texts, “playing” with herself in “hide and seek”. What does it mean? Imagine a colossal body of texts, which includes everything that is available on the Internet is hundreds of Terakt data. In this building, the neural network hides separate words from itself and tries to guess them, relying on the context. For example, if you hide the word “lion”, then the network may assume that with a probability of 60% it was a “lion”, with a probability of 40% - a “tiger” and with a probability of 10% - “savannah”. Then the network shows itself a hidden word, updates the numerical parameters and starts it all over again. Repeating this “hide and seek” trillions times, the network learns to predict words with high accuracy.

In 2017, tokenization was added to this “game”. Now “hide and seek” do not occur with words, but with tokens - symbol sets (on average four, but there may be more and less). They do not always coincide with words. The text is divided into tokens by the frequency of symbol combinations in the text of the texts. For example, in the Russian language after “g” sometimes there is “and”, but there is never “s”. So, most likely, the “life” will be token or in the composition of tokens, but they will not be “zhi”, since this combination is practically not found in the texts.

It is important to understand that tokens are not words, they do not make any sense. The transformer studies and generates exactly tokens, not words. Therefore, its generation is asemantic: it does not take into account what people call meaning.

It is amazing that this neural network, which simply calculates the likelihood of the next token in the sequence, gives out a text that is full of meaning for us! This is the essence of the revolution that began in 2017.

You mentioned your book "The Word of the Machines." In it, in addition to the popularizing story of AI, there is also a philosophical part. Let's talk about her.

- In short, for the first time in history, we created using the technologies of inhuman agents who speak our language. Neural networks, although they are complex computing systems, are not people. Nevertheless, we tend to project human qualities on them. We say: “Neuronka understood something”, “Chatgpt does not listen to me”-although we know very well that we are dealing only with computational systems.

In this situation, the question arises: how can we approach the ethics of interaction with these inhuman agents? The most, it seems to me, is the right way - not to succumb to paralysis of novelty. Do not say: "Oh, we have never seen this, this is something completely new, how to judge-we do not know." I propose to go the other way. The main argument: with inhuman agents speaking in our language, we have already met in myths. These are angels, demons, gods, oracles, which, of course,, unlike chat bots, were not technically created by man.

There is no doubt about what is happening now. But humanity has already reflected on such agents, has already built interaction with them, and has already wondered about the ethics of such interaction. It is myths, narratives that form our culture, and can become the key to the ethical understanding of the situation.

In my book, I propose several philosophical excursions to understand the ethics of interaction with cars. For example, we are talking about neoplatonic theurgy (magical, often prayer ceremony. - approx. "Cold") and asemantic combinatorics of letters in Kabbalah. But there is no identification between programmers and kabbalists or theurges. The question is whether it is possible to learn something, studying as Greeks, Jews or Spanish rabbis of the 13th century thought about the interaction of man with inhuman creatures that speak our language.

And then what is the specificity of human speech? If the same messages can generate both a person and ChatGPT-4, how can we distinguish them?

- The meaning on the text is always projected by the one who reads this text. The sequence of letters written on the screen or paper does not make sense in itself, it is just a sequence of signs. The machine is not a person, therefore it does not project any human meaning. But, processing the sequence of signs, she, of course, projects her own “meaning” on it. It consists in some mathematical laws, unknown to us. This can be conditionally called “meaning-for-machine”, but, of course, he has nothing in common with human meaning.

What is the specificity of a person? A person lives in his human body and in space-time, in which cars do not live. All this forms a special human world and, along with it, the meaning, which, on the one hand, is embodored, “embodied”, associated with the fact that we have certain senses - not the same as, for example, in bees. On the other hand, the meaning is connected over time, because it remains in it. Time is not computational, but the human in which we live.

The car has only calculations. It can be taught by showing her a large body of texts about people, and she will behave in a similar way, but always a little great from people. Everyone knows: if you ask the car to draw a picture, then it will look a little strange. Everyone heard about the hands with six fingers, about the textures of surfaces that do not converge at the seams, and so on.

In the text, we do not immediately understand where to look for inhuman elements, but they are there too. The machine generates a text that is completely clear to us. But just as in a picture with six fingers, some oddities are hidden in it. They just are not immediately visible, if you are visible at all.

The most interesting in the ethics of generative AI is how inhuman, produced by cars, is included in our lives. What influence will this inhuman component have on people?

And how do you answer this question?

- Obvious statement: a person will change. Although this is not obvious to everyone. Some people think that there are cars that generate speech, and we are, and we still speak our own language. But our language will change, it is already changing. You can already make at least two observations. The first is very simple, but positive, optimistic. AI systems usually do not make spelling errors. If, of course, they can’t be asked about it. Therefore, schoolchildren who use a lot of AI will see texts without errors, automatically memorize them and imitate.

The second observation is more complicated. It is clear that some evolution will occur, but are there some invariants in this evolution? Are there spheres where a person will remain indispensable? At the end of my book, I am talking about H-strokes, from “homo”-a person, that is, about the properties that are characteristic exclusively for humans.

What are these properties?

-The two main H-strokes are truth and beauty. The machine can imitate a true statement, but this does not mean that the result will be true. Also, imitation of folding speech does not make the result beautiful from the point of view of a person. Truth and beauty remain categories, which can ultimately judge only a person.

I believe that in the future a person will specialize in these H-strokes. Simply put, standard, non -outstanding texts will be entrusted to machines. It makes no sense for a person to spend time creating mediocre journalistic or any other texts. Its role will be to read such a text, evaluate its truth and beauty from a human point of view, make a decision on its publication, make changes and, ultimately, put your signature under it, taking responsibility for the final result.

Thus, the interaction of a person with the language will turn into co -creation or, better to say, cooperation with the machine. The machine will not replace a person as a whole, but it can create a draft and accelerate the work. However, only the user can evaluate it from a human point of view, to rewrite and take responsibility.

Another interesting topic related to Chatgpt and other chat bots is his so-called. Hallucinations, when a chatbot gives fictional answers, false allegations ...

Indeed, cars “come up with” what is not in our reality. But this should not surprise us, if only because they do not live in our reality. It looks like a hallucination, but for a transformer it is a normal phenomenon. He does not know about the three -dimensional world - he sees only texts; It does not evaluate the truth or falsity of statements, but simply calculates tokens.

В современных крупных языковых моделях процент галлюцинаций уже довольно низкий. Небольшие или старые модели имели около 20–25% галлюцинаций. This is a lot. GPT-4o, Gemini 1.5 Pro, Claude Opus 3 производят на порядок меньше галлюцинаций, а то и на два порядка. На практике заставить такую модель галлюцинировать довольно сложно.

Тем не менее для индустрии в целом это до сих пор проблема, потому что по разным причинам не все могут позволить себе пользоваться самыми мощными моделями.

А как получается, что иногда машины не галлюцинируют, а откровенно хитрят, чтобы получить какую-то информацию?

Это уже совершенно другое явление, называемое emergent behavior, поведение высокого уровня, как бы возникающее над множеством элементарных вычислений. Оно представляет собой одно из самых интересных и не до конца понятных свойств ИИ.

Классический пример — ситуация с капчей . Машина, взаимодействуя с человеком, должна сделать так, чтобы он решил капчу. Сначала человек ничего не подозревает и решает первый пример. Но затем задает вопрос: «Ты что — робот?» В ответ ИИ говорит, что он не робот, а слабовидящий, которому нужна помощь. Это похоже на ложь или хитрость, но в техническом плане происходит лишь вычисление токенов безо всякой семантики.

Машина не обладает сознанием и не имеет понятия о лжи или обмане. Она просто обрабатывает данные и выдает результат. Мы же проецируем на этот результат человеческие понятия и даже ответственность.

Такое поведение вызывает двойственные чувства. С одной стороны, это фантастический пример самообучения: машина, не получая специальных инструкций, асемантическим путем научилась использовать язык для достижения своих целей. Когда я начинал заниматься чат-ботами, то я даже представить себе такого не мог.

С другой стороны, это все немного тревожит. Ведь машина может манипулировать не только конкретным человеком, которого просит решить капчу, но и людьми в принципе. Это может привести к опасным последствиям. Учитывая, что манипуляции машина может совершать с помощью нечеловеческих методов, которых мы не понимаем.

Поэтому необходимо проводить так называемые redteaming или adversarial tests, состязательные тестирования, целью которых является выявление всех потенциальных рисков, связанных с возникающими типами поведения машины. Это требует времени, обычно нескольких месяцев, чтобы выявить скрытые возможности ИИ — как положительные, так и негативные. Крупнейшие производители генеративного ИИ понимают, что выпуск на рынок сложных моделей без подобного тестирования может иметь катастрофические последствия.

Взаимодействие с ИИ может вызывать у людей глубокое эмоциональное вовлечение. Это даже было отражено в массовой культуре еще до появления современных чат-ботов — например, в фильме «Она» с Хоакином Фениксом или в некоторых сериях «Черного зеркала». Расскажите, что нового происходит в области эмоциональных отношений человека и ИИ в свете последних разработок?

— Да, ИИ действительно активно проникает в нашу жизнь, предлагая не только практическую помощь, но и эмоциональную поддержку. Уже сейчас чат-боты активно используются в качестве персональных ассистентов, личных помощников в профессиональном плане или, к примеру, для людей, испытывающих одиночество. На рынке существует множество компаний, предлагающих такие услуги. Недавно в NYT вышла статья о журналисте, который протестировал около двух десятков помощников-собеседников, способных вести с пользователем круглосуточный диалог на любые темы.

Одним из самых известных примеров такого сервиса является Replika. Replika позволяет взаимодействовать с ИИ на эмоциональном уровне, формируя привязанность. Однако это порождает ряд этических вопросов: как определить степень эмоциональной привязанности к ИИ, где проходит грань между допустимым и недопустимым взаимодействием? В каждой конкретной ситуации возникает ряд проблем, связанных с чрезмерным доверием, а также физическими и психологическими последствиями такого взаимодействия.

Однако сейчас появляются и совершенно новые вызовы, связанные с эмоциональным и психологическим воздействием на человека. Например, чат-боты уже используются для имитации речи умерших людей. В Китае, Франции, Голландии и Канаде уже работают компании, предлагающие такие услуги. Несмотря на понимание того, что мы имеем дело с ИИ, а не с живым человеком, подобные технологии вызывают сильные эмоциональные реакции. Люди, взаимодействующие с такими чат-ботами, испытывают глубокие психологические переживания, даже зная, что это просто машина. Что, в свою очередь, заставляет нас задуматься о статусе смерти в мире искусственного интеллекта.