Translated material
Eliezer Yudkowsky. Serious stories
The most interesting works of literature and art are full of suffering, conflict and tragedy - without these elements, stories would not be as exciting and “serious”. On the other hand, if it were a question of what kind of life we want to live, all other things being equal, we would not be attracted by the prospect of a life full of suffering. But if people could build an ideal world, would they decide to get rid of all types of suffering at once? If not, why not?
In this essay, Eliezer Yudkowsky reflects on utopian transhumanism, suffering and pleasure, and the possible similarities and differences between a good life and an interesting story.
Every utopia ever created—whether in philosophy, literature, or religion—has been, to one degree or another, a place where you wouldn't actually want to live. I'm not alone in this important observation: George Orwell said much the same thing in his essay " Why Socialists Don't Believe in Happiness ," and I imagine many others have said it before.
If you read the “How to Write Books” books—and there are a lot of them because, amazingly, so many writers think they know something about writing—these books will tell you that every story must have “conflict.”
More precisely, “conflict” is if you read a softer book of this kind. Some authors put it more bluntly:
“Stories are about human pain” - Orson Scott Card.
“Every scene must end in disaster” - Jack Bickham.
In the years of my youthful stupidity , I took it for granted that authors were freed from the task of finding a real Eutopia , 1 because if you managed to build a utopia without any flaws... what would you write about then? “Once upon a time, they lived happily until the end of time.” What's the point of writing science fiction about a positive singularity if such a singularity would bring...
…the end of all stories?
This seemed a reasonable approach to the literary problem of Utopia, but something about this last conclusion raised a quiet, nagging doubt.
At the time, I envisioned AI as a kind of harmless genie that would grant people's wishes. So the conclusion seemed to make sense. If a problem appears, just wish it to disappear, right? Therefore, no stories. So I ignored that quiet, nagging doubt.
Much later, when I came to the conclusion that even a "safe genie" wasn't such a good idea , it seemed in retrospect that "no stories" might have been a useful indicator. In this particular case, “I can't think of a single story I would want to read about that scenario” might actually indicate a reason why “I wouldn't want to actually live in that scenario.”
So I suppressed my trained aversion to Luddism and theodicy - and at least tried to seriously consider this argument:
- A world where nothing ever goes wrong, where no one ever experiences pain or sadness, is a world where there are no stories worth reading about.
- A world you wouldn't want to read about is a world you wouldn't want to live in.
- In every eudaimonic life there must be at least a drop of pain. CTD.
On the one hand, it is obvious that we do not want to live the lives depicted in most of the stories that human authors have ever written. Think of the truly great stories - the pinnacles of their genre that have become legendary: The Iliad, Romeo and Juliet, The Godfather, Watchmen, Planescape: Torment, the second season of Buffy the Vampire Slayer, or that very ending in Tsukihime. Is there at least one story on this list that is not tragic?
In ordinary life we prefer pleasure to pain, joy to sorrow, life to death. And yet we seem to prefer to empathize with characters who are suffering, sad, and dead. Or maybe stories about happier people are frivolous , not artistically great enough to be worthy of praise - but then why is it that it is the unhappy stories that get praise? Is there any hidden benefit for us in this? No matter how you look at it, it's a puzzle.
When I was a kid, I couldn't write fiction because I made sure everything went well for the characters , the same way I wanted things to go well in real life. And Orson Scott Card cured me of this. “Oh,” I said to myself, “what I’ve been doing wrong all this time: my characters don’t suffer.” Even then, I didn't realize that plot microstructure worked the same way—until Jack Bickham said that every scene had to end in disaster. But I kept trying to pose problems and solve them... instead of making them even worse .
History simply cannot be optimized the way real life is optimized. The best story and the best life are generated by completely different criteria.
In the real world, people can live for quite a long time without encountering any terrible troubles - and at the same time everything will be completely normal for them. When was the last time assassins tried to kill you? It's been a while, right? So, does this make your life seem empty?
But on the other hand...
For some strange reason, when authors get too old or too successful, they go back to my childhood. In their stories, everything starts to go right . They stop doing terrible things to their characters - and, as a result, start doing terrible things to their readers. This appears to be a standard stage of Aging Author Syndrome. Mercedes Lackey, Laurel Hamilton, Robert Heinlein, even fucking Orson Scott Card - they all went there. They forgot how to hurt their characters. I don't know why.
And when you read a story by an Aging Author or a complete newbie - a story where everything just keeps going like clockwork - where the main character defeats the supervillain with a snap of his fingers, or worse, the supervillain gives up and apologizes before the final battle, and they're friends again ...
...it's like someone is scratching their nails on a chalkboard at the base of your spine. If you've never read a story like this (or worse, you've written one), consider yourself lucky.
This “grinding effect” - could it transfer from history to real life if you try to live your life in such a way that there is not an ounce of problems in it?
One possible answer is that what history really needs is not “disaster” or “pain” or even “conflict”, but simply complexity . That the problem with Mary Sue stories is that there is not enough effort in them , but at the same time, effort does not have to be accompanied by pain . Maybe this can even be tested.
Another answer might be that we're discussing a transhumanist version of Fun Theory here, so we might as well say "modify the brain to eliminate that nails-on-a-chalkboard feeling" unless we have a good reason to keep it. If this feeling is just a meaningless, random bug that hinders Utopia, it needs to be removed.
Maybe it really is worth it. Maybe all Great Stories are tragedies simply because... well...
I once read that in the BDSM community, the phrase “intense sensations” is a euphemism for pain. When I read this, it occurred to me that because of the way people are wired today, pain is actually easier to induce in them than pleasure. Although I'm speaking a little outside of my own experience here, I imagine that it would take several hours of work by a talented and experienced sex master to produce a good sensation comparable in intensity to the pain of one hard blow to the groin that a beginner could deliver in a couple of seconds.
While studying the life of the priest and proto-rationalist Friedrich Spee von Langenfeld , who confessed to those accused of witchcraft, I looked at what tools were then used to obtain confessions. There are no ordinary ways of making a human being feel as good as these tools can make him feel bad. I'm not sure that even drugs can do this, although I have no experience with them or with torture.
There's something unbalanced about it.
Yes, people tend to be overly optimistic in their planning. If losses were not perceived more painfully than winnings, then, given the way our brains are structured now, we would have gone broke a long time ago. According to experimentally discovered patterns, the loss of what you want (whether it’s 50 dollars, a mug of coffee or something else) causes about 2–2.5 times more severe suffering than a gain of similar size brings joy.
But we are talking about a deeper imbalance. The difference between sex and torture in terms of the ratio of “input effort” to “output intensity” is not just twofold.
If a person goes in search of thrills - in this world, with the current structure of man - it is not surprising that in the end he comes to the conclusion that in order to increase the intensity of sensations, pain must be mixed into pleasure.
If only people were constructed differently, so that pleasure could be evoked with the same intensity and variety as pain! If only it were possible, with as much ingenuity and effort as a medieval inquisitor puts into torture, to make someone feel as good as the victims of torture felt ...
But then... what pleasure would be analogous in intensity? The victim of skillful torture is ready to do anything to make the pain stop, and do anything to prevent it from happening again. Would the equivalent pleasure be one that overshadows everything else with the desire for only one thing: to continue and repeat this pleasure again and again? If a person is able to resist it, will it really reach that intensity?
There is another rule of writing: the story must scream . The human brain is very far from printed letters. To have any impact on the reader, every event and every feeling must be described ten times louder. Don't try to make your characters' behavior or feelings realistic . Especially if we are talking about you: you cannot simply accurately reproduce your own past experience. Because without exaggeration , everything will sound too quiet to rise from the page.
Maybe all Great Stories are tragedies because happiness cannot shout loud enough for the reader to hear it.
Maybe this is exactly what needs to be fixed.
And if you fix it... will there be any need for pain or sadness at all? And even the memory of sadness - if everything is already as good as it can be, and every fixable problem has already been eliminated?
Is it possible to simply remove the pain? Or, by removing the old utility function minimum, are you simply creating another minimum? Will then any pleasure less intense than 10,000,000 hedons become a new unbearable pain?
People with their current setup seem to have a tendency toward hedonic scaling. Someone who remembers what it's like to go hungry will appreciate a loaf of bread more than someone who has eaten only cake all their life. This was George Orwell's hypothesis , explaining why Utopia is impossible - neither in literature nor in reality 2 :
Man seems incapable of describing happiness—and perhaps even imagining it—other than through contrast... The inability of mankind to imagine happiness except in the form of relief from difficulty or pain poses a serious problem for socialists. Dickens was able to describe a poor family munching on roast goose and make them look happy. But the inhabitants of perfect universes do not seem to have spontaneous cheerfulness; in addition, they tend to be somewhat off-putting.
For an expected utility maximizer, adding a trillion to all values of the utility function is meaningless: mathematically, the result is literally the same utility function. A utility function describes the relative intervals between outcomes—that's what it is in the mathematical sense.
But the human brain is structured differently: it has separate neural circuits for positive feedback, and separate ones for negative feedback, and different types of both positive and negative reactions. There are people who “suffer” from congenital analgesia - a complete inability to feel pain. And I have never heard of a lack of pleasure becoming intolerable to them.
People with congenital analgesia need to regularly and carefully check their body to see if they have cut themselves or burned their finger. Pain serves a specific function in the structure of the human mind...
But this does not prove that there is no alternative capable of performing the same function. Is it possible to remove pain and replace it with a urge to avoid certain actions that will remove the unbearable subjective quality of pain? I don't know all the Laws that apply in this area, but I would suggest that yes, you can: you can replace this side of yourself with something more like an expected utility maximizer.
Is it possible to remove the human tendency toward hedonic scaling—to remove the addictive effect, so that each new portion of roast goose is as satisfying as the first? I would suggest that it is possible. This comes dangerously close to removing Boredom, and it ranks next to Empathy, which is absolutely irreplaceable... But if the old way of getting pleasure works equally well every time, that doesn't mean you won't be motivated to look for new, better ways.
Is it possible to make every roast goose taste as delicious as it would in comparison to the famine, but never experience the famine itself?
And if literally the worst thing you've ever experienced is the pain of a speck in your eye, can you prevent it from becoming the equivalent of torture?
Such questions are beginning to go beyond my understanding of the Laws, but I would suggest that the answer is yes, it is possible. My experience is that once you have mastered the Laws, you usually understand how you can do things that now seem strange.
As far as I know or can guess, David Pearce (author of The Hedonic Imperative) is probably right about feasibility when he says 3 :
Nanotechnology and genetic engineering will abolish the suffering of all sentient beings. This abolitionist project is extremely ambitious, but technically feasible. It is also instrumentally rational and morally compelling. Metabolic pathways for pain and illness evolved because they helped our ancestors' genes adapt to their environment. They will be replaced by a different type of neural architecture - a motivational system based on inherited gradients of bliss. States of sublime well-being will become a genetically programmed norm of mental health. The world's last unpleasant experience is predicted to be a precisely dated event.
Is this what... we really want?
Just wipe away the last tear and that’s it?
Is there any good reason not to do this, other than status quo bias and a handful of tired rationalizations?
What is the alternative , or alternatives?
Leave everything as it is? Of course not. No God designed this world and we have no reason to think that it is truly optimal in any respect. If the amount of pain in this world is not excessive, then it would be insufficient - but the latter seems unlikely.
But maybe...
Is it possible to remove precisely the unbearable parts of the pain?
Get rid of the Inquisition. Leave that pain that warns: don’t put your finger in the fire. Or the one that says: you shouldn’t have put your friend’s finger in the fire. Or even the pain of parting with a loved one.
Try to get rid of the pain that grinds and destroys the mind. Or make the minds themselves more resistant to damage.
It may be possible to have a world where there are broken legs and even broken hearts, but there are no broken people . There is no child sexual abuse that creates more abusers. There is no such thing that people are overwhelmed by boredom and exhausted by petty everyday life - to such an extent that they think about suicide. There is no senseless endless grief, like dying of hunger, or AIDS.
And even if breaking a leg still seems too scary...
Would we be less afraid of pain if we were stronger, if everyday life did not take so much of our strength?
This could be the alternative to Peirce's world. If there are others, I haven't seriously considered them yet.
You could call this the path of courage - the idea is that if you eliminate the destructive types of pain and strengthen people, then what remains will not be so terrible.
A world in which there is sadness, but not the large-scale, systematic, senseless grief that we see on the evening news. A world where pain, if not completely eliminated, at least does not outweigh the pleasure . Stories could be written about such a world, and its inhabitants could read them.
I generally tend to be conservative when it comes to removing large parts of human nature. Не уверен, сколько значимых частей можно вырезать, пока эта сбалансированная, конфликтная, динамическая структура не сколлапсирует во что-то более простое — например, в максимизатора ожидаемого удовольствия.
И потому я признаю, что именно путь мужества мне ближе.
С другой стороны, я не проживал ни одного из этих путей.
Может, я просто боюсь мира, настолько чуждого, как Анальгезия — не правда ли, ироничная причина, чтобы избрать «путь мужества»?
Может, мне просто кажется, что путь мужества несёт менее радикальное изменение. Может, я просто не могу прочувствовать более значительный разрыв.
Но ведь «изменить мир» — это движущаяся мишень.
Если ребёнок вырастет в менее болезненном мире — если он никогда не жил среди СПИДа, рака, рабства, поэтому не знает их в качестве ужасов, которые были триумфально побеждены — если он не чувствует, что мы «уже закончили» или что мир «изменился уже достаточно»…
Сделает ли он следующий шаг, попытавшись устранить невыносимую боль разбитых из-за невзаимности сердец?
What then? Наступит ли момент, когда «Ромео и Джульетта» просто будет казаться всё менее и менее актуальной, всё больше и больше реликтом какого-то далёкого и забытого мира? Наступит ли такой момент в трансгуманистическом путешествии, когда вся работа системы негативного подкрепления будет восприниматься не иначе, чем бессмысленный отходняк, от которого пора пробудиться?
И если так, то есть ли смысл оттягивать этот последний шаг? Или стоит просто отбросить наши страхи и… отбросить всё, что нас пугает?
I don't know.
* * *
Оригинал: Eliezer Yudkowsky, Serious Stories (2009), CC BY-NC-SA 3.0 . Перевод: К. Кирдан для проекта Reducing Suffering и LessWrong на русском . Illustration generated by AI.