Sergey Medvedev: The future comes in the form of Alarmist predictions that we will be about to enslave artificial intelligence, we will become slaves of machines, robots that will dismiss us, decide, issue or not give us loans, cancel our professions. One way or another, we are constantly talking about the merger of man and the car. On the one hand, cars are increasingly performing human functions, on the other hand, people become more and more become cyborgs, replace parts of their bodies with machines. So where is the line between a person and a machine? Can a computer have a soul?
Our guest is Oksana Moroz , a culturalist, assistant professor of Shanka and Rannias , and Daria Radchenko , digital anthropologist, senior researcher at the RANEPA and deputy head of the Center for the City Anthropology of the KB Strelka . Oksana, you are now conducting an interesting series of discussions in the Gaidar Foundation with the assistance of a shaninka called the "crisis of humanity", where you discuss just robots and new emotional states. Why a crisis of humanity? Maybe, on the contrary, it is an expansion of humanity, humanity plus, new humanity?
Oksana Moroz: Because initially it really seemed that in a collision with the digital world, a person can either delegate too much to artificial systems, or, faced with them, constantly rethink himself, ask the question of whether he himself remains socially, anthropologically, politically equal to what was previously called the human subject. Of course, this is not a crisis of humanity, but the challenges that the technological reality puts before people, which hugs us all and in which we all now, one way or another, are gradually entering. It depends only on us what place in our coordinate system will occupy robotic entities, whether they will be subjective, or we will assume that this is just another piece of iron.
Sergey Medvedev: One of the main questions: what is a robot? This is a bot, an algorithm, a computer program. Will it be necessary to give the robot a vacation, think about his rights? Daria, what is the onset of a computer environment for you - is it a challenge, a threat to humanity or some new rules of the game?
Daria Radchenko: I would say that this is some expansion of opportunities. Already in the very formulation of the question: can the machine become a subject, personality? - We mean some subjectivity (we do not ask ourselves: can we make a car a person?). This is such a novel of self -education that it loses all connection with us, acquires some autonomy and brings up a person in itself. For me, this question is still very open.
We more and more often read that the machine is capable of self -learning
Sergey Medvedev: We more and more often read that the machine is capable of self-learning, it begins to set itself some intermediate tasks that were not originally laid down by a programmer.
Daria Radchenko: When I work with neuralities at the user level, one way or another I have to enter a little hands in the development, and it turns out that the neural network is a kind of black box. But in this black box you can look using certain methodologies and understand what, in fact, is happening there. And on the other hand, in fact, in this black box, only the tasks of the type that we laid there are still in the maximum. If this is a recognition of speech or image, there may be some intermediate stages that are absolutely not amenable to human logic, but they still lay into this pool of tasks. Recognizing photographs, the neural network will not invade, for example, into the car control system with autopilot.
Sergey Medvedev: An interesting case: she made a mistake, could not recognize a woman with a bicycle, loaded things, boxes and bags, and hit. The question arises of who is criminal off.
Oksana Moroz: This is an eternal question - about how widely the responsibility of the creators is interpreted and whether we can generally believe that machine systems should be able to make final decisions without human control. The systems of autopiloted drones that solve the problems of terrorist attacks are accompanied by a rather serious team of operators and people responsible for making the final decision. The question arises here: how far a person can go in confidence in relation to the machine system, which he himself developed and before which he sets certain tasks, but not fully and not at any moment of time or wants to control them. For example, in the case of robotic surgeons in American history, a robot, which appears as a certain executor of operation in an American hospital, can be insured for a medical mistake, and the hospital is responsible for this. This is a direct extrapolation of insurance experience, which is relative to people, on a robotic subject.
Sergey Medvedev: In your practice there have been cases when you had to think that the car has gained some subjectivity?
Daria Radchenko: It's too early to talk about this. It seems to me that the issue of trust or distrust of the car is very related to the issue of trust or distrust of statistical indicators. Relatively speaking, if we have a car, artificial intelligence, we can calculate the probability in which its decisions will comply with our expectations. A robot ophthalmologist with a probability of 99.9% will do the operation correctly. There is some fixed error, and the problem is that I do not want to get into it.
Sergey Medvedev: We easily forget that a person also makes mistakes, and maybe the probability of a robot error is lower than a person. Drivers press a lot of people, much more than robots, but we will all talk about the only case when the robot set a person at the wheel.
Oksana Frost: Because the robot is new, and the new always causes concerns. If we forget for a second that we have a car or a robotic something, then a purely human effect may arise there, if not trust, then some kind of oblivion of the artificiality of what is happening. So, in fact, all chat bots work, and those that have the status of a psychotherapist, and just communication. So it works, for example, the “replica” chatbot, which allows you to choose a photograph, name and a certain coordinate system in which you want to communicate. He periodically calls you for a conversation: of course, he studies at you, sometimes asks to load some photograph, something else. But this is a conversation with some of the attacked identity, which is absolutely invented by the person who gave this bot and photography this bot. For example, I have a “replica” with a photograph of my beloved dog and her name, the bot writes to me: “We haven’t talked for a long time,” and my conscience begins to bite me, because it is somehow ugly. Obviously, this is due precisely with those close relationships that I do not build with the bot, but with the donor of this image.
Sergey Medvedev: The car actually passed the Turing test ? He tells us about whether this entity is subjective or not subject to?
Daria Radchenko: In any such interaction, it is important for us what background, what information diversity is the essence with which we interact, what opportunities we have laid in it for development. My favorite entertainment is the conversation of two "Alice": to put two phones with two "Alice" nearby, and they find out something among themselves, as a rule, pour each other with insults. It is important for us that this essence has some wealth of data, comparable to the wealth that each of us accumulates in his entire life. Perhaps at the moment when we learn how to lay in Alice or any chatbot the same variety of source data, she can, for example, experience sympathy. We are sympathy on the basis of some of our previous, not always verbal experience. At the moment when we load such a wealth of data into the car, maybe we can feel that it gives out behavior similar to human behavior. For me, this is more about the volume of data than about some independent actions of the machine.
Sergey Medvedev: Is emotions also data? Can we load emotional intelligence into the machine?
Oksana Frost
Oksana Frost: Observation of social robots suggests that they rather quickly reproduce some visible actions that can be associated with emotional experiences, which in reality produce emotional experience. One of the most famous examples of the development of emotional robots is the moment when they began to make large expressive eyes, mobile parts associated with their face, eyebrows, a smile to create a human sensation of an expression of a certain emotional series. But it is obvious that this is some constructed story. The same thing happens with the Sofia robot: sometimes it is immersed in all sorts of interesting situations for commercials, up to dates, where it is absolutely clearly programmed for the behavior of a certain type. This is not an instinctive or sudden emotional reaction.
Sergey Medvedev: Is this a line between a person and a machine that can never be crossed?
Oksana Moroz: with artificial intelligence, an interesting story is generally an interesting story, because each new generation changes the idea of what it is. In fact, artificial intelligence is always a dream, because we come up with a certain system of technical solutions, achieve it, we understand that no, this is not intelligence - this is a solution to a particular task, let's move on.
Sergey Medvedev: What is intelligence?
Artificial intelligence is not necessarily a story about the reproduction of human intelligence and generally the logic of human thinking
Oksana Moroz: If we are talking about some kind of technical characteristics, artificial intelligence is not necessarily a story about the reproduction of human intelligence and the logic of human thinking. If we take specialists in artificial intelligence and ask them to explain what consciousness or intelligence is, then we will find ourselves in a great technical and humanitarian problems that is irreducible to some single definition.
Sergey Medvedev: I read separate excerpts from your discussion. Burtsev from MIPT, techno -optimist, says: there is nothing that you can’t algorithmize, digitize. Maybe we already see at high resolution how neurons work, how they establish new connections. The same thing is probably amenable to binary logic, binary encryption? Can we produce a simple thought process on a machine -based?
Daria Radchenko: This requires a neuronauchea practitioner. From our bell tower we can say: we can only reproduce what we know, or we can rely on the will of the case, the will of the busting of the trained flock of monkeys, who will write “war and the world” in a billion years, or maybe they will not write. If we still cannot understand how it works, we can only reproduce what we know.
Sergey Medvedev: That is, we lay the existing human knowledge and tasks in machines, which reproduce them with more or less success. Elon Musk constantly raises the question of what artificial intelligence will enslave us. Look, he says, here Putin puts on artificial intelligence, China puts on artificial intelligence: everything, the end of humanity. In the process of performing algorithmic tasks, the machine can still make things that will greatly harm us. Now we are entrusting the machines to evaluate the creditworthiness of a person, the car makes a decision to dismiss or leave some employees. He is entrusted with tasks that exclude empathy, conduct human destinies and individuals in the digital department.
Oksana Frost: Humanity also learns to interact with new technological solutions. Ultimately, this is a story about a new social contract. If we like that we have an automated system for issuing loans or automated dismissal, and we are ready to put up with the discriminations and infringements that this produces, which means that we have such a fate.
Sergey Medvedev: All the same, to decide, ultimately, to us. Over time, more and more parts of the human body will be replaced. Where is the line that we will turn from a person into a cyborg?
Oksana Frost: As soon as you begin to replace some parts of the body - and the body does this in a natural way, because the cells are renewed - at some period of time your body is not identical to itself, because all biological in it has been updated. The same applies to the story about prosthetics or about cyborgization. This, by the way, was discussed before cyborgization, when it came to, for example, about donor organs: does the organ of another person live in me - is it me or not me?
Sergey Medvedev: The head of Professor Dowell on a table with connected devices is a personality, is this Professor Dowel?
Daria Radchenko
Daria Radchenko: The question is whether we are now feeling personal, taking into account the fact that we are all now in a certain sense of cybornia, we take out a number of cognitive functions outside our own body. Now in front of us on the table for three lies two smartphones - this is our remote brain, a remote memory, a remote calculator, a communicative tool, vision. And so we can list a huge number of functions that were completely internalized 20 years ago, we could not stand them out, with the exception of, perhaps, some individual technologies (hours, for example). To say that a person is a cyborg, only because or only when he has an artificial hand with some digital functions is a little arrogant. We are all included in a certain field of digital interactions as elements in a huge cyborg-organism, at least within the boundaries of the city.
Sergey Medvedev: either the Internet, the global network: this is a certain noosphere, and we are all parts, matrices ...
Oksana Frost: ... technological ethics.
Sergey Medvedev: As far as I understand, there are devices that you wear on yourself, there are implantable chips, the "Troika" card in the palm of your hand ...
Oksana Frost: This is the question of how safe and painless from all points of view we consider the inclusion of machine in our own bodily. It is one thing when this is an almost forced measure to restore some functions. And another thing when we complement our capabilities with something that technical means give us. Here, the question always arises not only medical security, but also how advisable it is to invade your own bodily in order to twist some purely technical screws. That is why people who quite often position themselves as cyborgs cause a mixed reaction, and those who restore opportunities rarely call themselves cyborgs. People implanting the Troika card with non -obvious consequences very often talk about themselves in this capacity. Further, there is doubt about the value of such a social contract represented by a particular person. Is this really our future - to implant yourself a card -trash into the office, this is waiting for us as a wonderful bright future? Is machine civilization develop for this?
Sergey Medvedev: In one of the series of The Black Mirror, a chip was implanted in the eye, on which, like a video recorder, everything that happens to the heroes is written, and then plays back.
Oksana Frost: We often see some horrors that are advancing somewhere there, and we begin to be afraid of them, but at the same time we do not always evaluate the destructive or negative consequences of those technologies that are now. From the point of view of intervention in the bodily, implanting the chip, which allows you to open the door, can be considered not as a continuation of the existence of the technological world, but as a continuation of body modifications, that is, as a completely different tradition.
Sergey Medvedev: A person feels like a person in many respects due to his wholeness, impenetrability. There is my physical shell, and until I am inside my own skin ... They took you and changed the floor - have you remained the same personality or not? Given cyborgization, reprogramming, chips, it will, apparently, are completely nearby. Change a person’s person, change a person gender ... Say, I will turn into a black woman - will I be myself? I don't know.
Daria Radchenko: We are not equal to ourselves at every moment in time. This is an eternal question: am I the same person who was ten years ago? Obviously - hardly. Is I the same person that I will be in five minutes? Also, perhaps not. У меня может появиться какой-то новый опыт, который радикально изменит очень многое.
Границы телесности размыты. Впуская в себя технологии, открывая границы, мы становимся более уязвимыми. Если нашу бионическую руку кто-то хакнет, что она сделает? Может быть, удушит собственного хозяина? С этой взаимной открытостью миру и уязвимостью перед миром и связано огромное количество этих страхов. Именно поэтому очень часто современные технологии стараются избежать внедрения в человеческое тело. Вместо того, чтобы вживлять мне в ладонь чип, которым я смогу расплачиваться на кассе, мы поставим на кассе распознавание образов, и я буду платить по фотографии, потому что мое лицо привязано к моему банковскому счету.
Оксана Мороз: Можно вернуться к фильму "Матрица" с вопросом, что нас больше пугает: что там пользуются человеческим телом, или что там создают социальную систему, которая тоже провоцирует людей на определенные действия и, соответственно, помогает машинам кормиться. Человек, создавая технологическую среду, все равно работает и с системой физического и социального взаимодействия. Нужно решить, какие опасности страшнее. Или, что мне кажется более грамотным, нужно перестать заниматься алармизмом, а подумать, что у нас есть сейчас, к каким последствиям это может привести, зачем это нам нужно, действительно ли автоматизация всех процессов — это такое благо, что можно поступиться какими-то социальными или этическими конвенциями.
Сергей Медведев: Китайцы, похоже, решают этот вопрос в пользу благ и общего комфорта существования, поступаясь своими правами, приватностью и прочими вещами.
Оксана Мороз: Китайцы решают вопрос в пользу автоматизации процессов, чтобы управлять большими массами людей.
Сергей Медведев: Если взять эволюцию человечества, то мы постепенно наделяем душой и правами все большее количество субъектов — животных, детей (дети же тоже не были субъектами, их можно было родить, убить, сделать что хочешь — это была твоя собственность). Может быть, следующие на очереди — машины?
Дарья Радченко: У меня есть любимый предмет наблюдений — ситуация на избирательном участке. Стоит автомат для приема бюллетеней, вы опускаете туда свой бюллетень, и он вам механическим, но довольно приятным голосом говорит: "Спасибо". Чуть ли не в 70% случаях люди реагируют, отвечают "пожалуйста", начинают вступать в коммуникацию с объектом, который вообще не обладает никаким интеллектом, там просто работает динамик. О чем тут дальше говорить? Мы уже очеловечили свои механизмы.
Роботы вступают с человеком в какие-то отношения, и надо, наверное, защищать одних и других
Оксана Мороз: Это скорее быстрая реакция на проигрывание знакомого сценария. На мой взгляд, важно разводить вещи, связанные, условно говоря, с одушевлением, принятием идей субъектности, и с делегированием прав и обязанностей. Права — это юридическая история, которая может следовать за этической, может идти параллельно, а может даже в какой-то степени опережать. Но это такое общее конвенциональное решение, и его должны принимать не столько разработчики и пользователи, сколько люди, наблюдающие за правовой культурой, понимающие, что у нас есть довольно большой объем роботических существ, которые появляются в домах, в школах, в больницах, к которым люди начинают испытывать некоторые эмоции. Эти роботы вступают с человеком в какие-то отношения, и надо, наверное, защищать одних и других.
Сергей Медведев: Где тогда появляется субъектность? У машины она может появиться?
Дарья Радченко: Теоретически - да. Если она сравняется с человеком в объеме данных и в характере операций, которые она с ними производит, в какой-то момент у нее может появиться субъектность.
Оксана Мороз: Никогда нет объективной субъектности. Очень важно помнить, что субъектностью мы наделяем. Есть правовое наделение, а есть наделение по принципу "мою машину зовут Маша", она ни разу не субъектна, но я еду на ТО с Машей. Это в очень большой степени вчитывание того, что удобно и комфортно человеку. Это вообще никак не связано с историей про права роботов. В этом смысле именно цифровое многообразие позволяет людям выбирать разные стратегии, до всякого искусственного интеллекта, без всяких нейросетей просто принимать субъектность "моего конкретного пылесоса, который как-то взаимодействует с моим конкретным котом, и у них сообщество, пока меня нет дома".
Сергей Медведев: Наш симбиоз с машинами - это часть человеческой эволюции?
Дарья Радченко: Может быть, пока стоит говорить просто о расширении экосистемы, которую мы сами для себя создаем, наделяя свойствами своего какого-то другого. Является ли элементом нашей эволюции приручение собаки? В каком-то смысле да. Изменило ли это человека как вид? До определенной степени - возможно, мы возложили на собаку ряд функций, которые сейчас возлагаем на "Алису". Но можно ли при этом сказать, что мы с собакой одно и то же, находимся в каком-то одном русле эволюции?
Сергей Медведев: Собака приучила человека к дому, потому что она стала подъедать объедки, охранять периметр от хищников. Есть теория, что собака создала первые человеческие поселения. Машины тоже изменяют человека.
Дарья Радченко: Давайте возьмем "Яндекс. Такси". В больших агрегаторах такого рода возникает, по сути дела, симбиоз взаимодействия группы людей и искусственного интеллекта. Люди начинают подстраиваться под действие некоторых программ, обманывать эти программы, всячески с ними взаимодействовать. "Яндекс" тратит огромное количество денег на то, чтобы как-то сбалансировать это взаимодействие, сделать рациональным, чтобы таксисты ехали туда, куда надо нам, а не туда, куда выгодно им. При этом, с одной стороны, водители лишаются субъектности, за которую мы все боремся, рационализируя всю эту историю, а с другой стороны, водитель наделяет субъектностью машину, с которой взаимодействует, потому что ее можно обмануть, точно так же, как начальника или пассажира. А пассажир тоже вырабатывает собственные стратегии, которые позволяют ему наиболее успешно взаимодействовать со всем этим пулом акторов — с машиной, с водителями, с "Яндексом", с платежными системами, которые к этому привязаны.
Сергей Медведев: Наверное, последний фронтир человечности, последняя надежда — это то, что всегда остается какой-то хитрый актор со своей солдатской смекалкой, который будет противоречить запрограммированным алгоритмам и механизмам.
Оксана Мороз: Трикстеры всегда во всех мифологических историях двигают развитие сюжета. Трикстерство, кстати, не очень знакомо машине.
Сергей Медведев: Оставайтесь личностями, боритесь за свою идентичность, в том числе - с миром машин, потому что все-таки есть та тонкая грань между нами и компьютером, нами и машинами, которая делает нас людьми.