
Earlier, we talked 1 about how multifaceted the mediator dopamine 2 is, with what phenomena it is associated with, but the question of causal relationships between the activity of dopamine neurons and a certain behavior remains fundamental. More or less satisfactory answers to such questions can give a method of local stimulation of brain structures. With electrical stimulation of the brain in a certain area, all cells are irritated simultaneously: it can be exciting neurons and brakes, there can be cells that direct their signal to other parts of the brain, and there may be internet. It can also be conducting paths belonging to cells that are far from the place of stimulation and are not related to the field of stimulation. How to achieve point activation of only dopamine cells or their endings?
Genous engineering comes to the rescue. Thanks to the impressive successes of molecular biology, now in almost any specific area of the brain, only dopamine neurons or their ending can be aimed and pointed. The main thing is that they are present there.
How does this work? In the genome of an experimental animal (most often these mice), using complex manipulations using viruses, a gene of unicellular green algae is built - an encoding protein called channel rhodopsin (in the figure below - KR). This protein forms a special membrane canal in the cell. A feature of this channel is that it is sensitive to the light: under the influence of a certain range of light waves, this channel quickly opens. When the light is turned on, the channel is open, the light is turned off - closed. The opening of the channel, in turn, leads to excitation of the cell: nerve impulses occur in the cell. Thus, we can activate the cage by turning on the light, and stop activation, turning it off. And this can be done with a high temporary resolution, that is, quickly enough so that stimulation can be correlated with any events occurring surrounded by the animal, as well as with any aspects of its behavior.

But how do dopamine cells activate? The fact is that the gene encoding the canal is built into the genome of an experimental animal in a strictly defined place: following a genome working only in dopamine neurons (this gene encodes a key enzyme responsible for the synthesis of dopamine). Therefore, the channel rhodopsin gene is included only in those cells in which dopamine is synthesized. As a result, it turns out that photosensitive channels are formed and integrated into the membranes of only dopamine neurons. And when, with the help of a fracture implanted into the brain of the animal, they illuminate the studied area of the brain, only dopamine neurons 3 are activated.
This method of selective stimulation of certain cells using light is called optogenetics . Here, in general terms, we described only one of the options for the method. Also, with the help of optogenetics, you can selectively brake (turn off) cells of a certain type, moreover, it can be stimulated strictly defined inputs to the studied cells (one cell can receive dozens of heterogeneous inputs), as well as the end of these cells in distant structures. And all this can be combined! Moreover, with maximum temporary and spatial resolution - that is, with very high accuracy both in time and in space: in a strictly defined place of a cell of a certain type, you can turn it on and off instantly.
Having received such a complex and unique tool, researchers with double enthusiasm took up the solution of problems that had long been waiting for their turn. One of the first such issues was, of course, in what the role of dopamine in reinforcement processes. The most indisputable, definite and unambiguous result of research in this area was the affirmative answer to the question of whether dopamine neurons are involved in the behavior of self -stimulation. (You probably remember this popular story about animals that, before exhaustion, shake and press on the pedal, which includes the electrical stimulation of certain areas of the brain). This question was waiting for an answer from the time of opening the phenomenon of the self -stimulation, and now by researchers from several laboratories it has been proven that the activation of the dopaminergic neurons of the average brain is enough to form and maintain this behavior in rats and mice 4 .
But the problem is that the resolution of the issue of criticality of dopamine for self -stimulation does not explain the nature of the phenomenon itself. Why does the animal constantly stimulate its dopamine neurons? According to many researchers, the behavior of the animal itself already proves the “ attractiveness of this stimulation for the rat ” 5 : by pressing the pedal, the animal gets incredible pleasure. Otherwise, why does the rat return to the pedal again and again? Widely discussed among scientists (and not only) the specificity of pleasant sensations. As a result of scientific discussions, an agreement was formulated that we mentioned earlier as one of the main hypotheses on the role of dopamine in behavior: dopamine is a nervous “currency” of any reward. Typically, the textbooks are followed by food and sex, and in some popular science books they write about “sensual pleasures”, which can be born of dopamine as a “gingerbread” that supports us exclusively to primitive pleasure. But here we must emphasize: we are talking about all rewards regardless of our attitude to them (from hunger swallowed with bones, and perhaps with giblets of herring to admiration, reading a favorite book, solving an interesting task, a gentle maternal touch or the long-awaited father’s approval, a smile of a lover, and satisfaction from the form of his reflection in a mirror or a mirror or a mirror or suddenly The discovered meaning of life). And already after you can reflect on the words of Marseille Zhuando: “ pleasure is a universal stumbling block. You rise it or it belittles you. What I do of it, and what I promise to make him out of myself clearly characterizes me - dishonors or glorifies, condemns or justifies, destroys or saves , ”and with pleasure to disagree with them.
“ The concept of awards ... carries a huge burden of emotional associations and interpretations-desire, fear, wine, duty, ban ,” said the bichevorist scientist Karen Pryor, the author of the book “Do not growl on the dog! A book about the training of people, animals and oneself. ” Is it worth it to take on this load? Objectively in the behavior of self -stimulation, we observe repeatedly repeated actions, reinforced by the stimulation of dopamine neurons.
With this approach, no subjective sensations are fundamentally put behind the concept of “reinforcement”. “ Reinforcement is something that, which happens simultaneously with the action, leads to an increase in the likelihood of a repetition of this act in the future .” Please note that such an approach implies a completely different view of the nature of behavior: if in the first case the action was determined by its goal - receiving pleasure, then the emphasis is on the predestination of the action of previous reinforcements.
Can the optogenetic stimulation of dopamine neurons of the midbrain with reinforcements serve? The study of Elizabeth Steinberg with co -authors, published in 2013 in the journal Nature Neuroscience , is the “causal connection of error of prediction, dopamine neurons and learning” 6 . From the heading of the article it is clear that the authors make an application for the fact that they received a decisive certificate in favor of the hypothesis of the role of dopamine in learning processes. The “error of prediction”, which is discussed in the name, is strictly associated with the activity of dopamine neurons for neurophysiologists. It literally refers to the category of errors on which they study. Steinberg and co -authors found that a dopamine signal encoding such an error in the prediction of remuneration is the cause of learning.
But in order to prove that the activity of dopamine neurons encoding the error error is really the cause of learning, it is also necessary to show the effect of a negative error: when the repeating no longer activation, and a short -term pause in the activity of neurons immediately after the condition of a conventional stimulus should theoretically lead to an increase, but to a weakening of the association of this stimulus with remuneration. However, this task is not easy: eliminating reinforcements, we get a quick extinction of learned reactions. A group of scientists under the leadership of Jeffrey Shoenbaum 7 used a sophisticated, but at the same time elegant methodological technique. Such techniques in English -speaking literature are usually called Elegant , which for the ear of a Russian -speaking person is somewhat unexpectedly applied to research. To try to understand his essence, we will make a trip to the store for groceries (and let's say that we do not have a rich experience in shopping). So, we have an electronic store of the store, which gives a good discount seemingly all purchases. Arriving at the store, we find that the action takes place on this day: our favorite chocolates are sold with a very attractive discount. We pretend to be in the mind that, along with a discount on the map, it comes out almost for nothing-and we collect this yummy for a couple of months in advance, because the time is now ... But when we pay for the purchase at the checkout, they say to us: “And for the goods on the discount no-e-e-time!” Well, okay, but we were not going to spend so much money.
That's what the rat feels like this in the situation created by the experimenters of high expectations. At the beginning of training, in response to three different conditional stimuli, presented separately, one portion of remuneration is issued; At the next stage, two of these incentives are presented simultaneously, and the animal also receives one portion of remuneration, the third stimulus, just as at the first stage, is presented separately with the same portion. The most interesting thing is found at the final stage, when all the same incentives are again presented separately. Now it turns out that the incentives that were presented jointly at the previous stage are much less attractive to the animal compared to the incentive, which was always presented in isolation. The experimenters interpret these results as follows: when the conventional incentives are jointly presented, the situation of high expectations is created (approximately as in our store, when we summarized the benefits from the action and the action of the discount card) and when these expectations are not met, the stimuli are gone to former attractiveness, and this loss is determined by the negative error of the prediction (expected a double portion, and received one).


What is attractive to this behavioral paradigm for experimenters? As you can see, it does not have a rapid reduction in explicitly, which usually leads to a negative error of predicting the remuneration, but it can also lead to some other implicit effect of exposure. And here the remuneration continues to serve, all reactions are preserved, and it would seem easier to interpret ... Using this approach to study the functions of the dopamine system, Shoenbaum and employees, however, did not begin to be lost rats - they gave them a double portion of freshly smelling sweet balls, but all the time, while the rat was eating the additive, after the rat. Her dopaminergic neurons were optogenetically inhibited. As a result of the testing, it turned out that the incentive, which was presented separately, is more attractive to the animal, despite the fact that with jointly presented incentives there was a double reward (discounts were summed up) - it means that the inhibitory effect of optogenetic stimulation acted as a negative error of prediction.
Trying to clarify the nature of the self -stimulation at the new methodological level, the researchers led by Satoshi Ikemoto decided to experience the determination of rats to achieve stimulation at all costs. No, they did not send them for living water or young apples, they simply changed the stimulation parameters. They changed the temporary interval between presses and reinforcements, as well as the number of necessary presses on the pedal to obtain one salvo of light stimulation of dopamine neurons 8 . It turned out that the behavior of animals working for the sake of stimulating dopamine neurons is radically different from the behavior of animals that press the pedal to obtain food reward. What are the most striking differences? While a tiny piece of food for a few seconds leads to the cessation of presses on the pedal, the short -term excitation of dopamine neurons, on the contrary, accelerates the initiation of the next press, but this was only enough for one second. But if you increase the interval between reinforcing stimulations - up to three seconds, the mouse quickly loses interest in the pedal, and even at ten seconds, it stops clicking on the pedal, while in such a situation a hungry mouse presses with increasing pressure. The authors, who throughout the text of the article, light stimulation of dopamine neurons with reinforcement , as a result, come to the conclusion that this is not quite reinforcement or even not at all reinforcements. They believe that the activation of dopamine neurons facilitates the initiation of action (helps to act) and this motivational effect is very short. Noticed? Reinforcement changed to motivation! Thus, a thorough analysis of behavior during self -stimulation leads to conclusions not in favor of the great significance of the role of dopamine in reinforcement processes. And it turns out that the third hypothesis about the specific role of dopamine in behavior finds confirmation in the described work: dopamine mediates motivation. Of course, the concepts of “motivation”, “reinforcement” and “reward” go hand in hand. Sometimes in natural behavior they are difficult to use and, moreover, inseparable at the level of incentives: motivating incentives reward, and rewarding incentives motivate. Which of us did not feel a surge of strength after sincere praise? But when we discuss the learning of a specific movement, for example, pressing the pedal, this very movement is a watershed between motivation and reinforcement: motivational processes are critical before the movement, reinforcement processes - immediately after.
Kelly McGonigal in the book “Willpower. How to develop and strengthen ”asked by questions:“ What if the rats of Olds and Milner stimulated themselves before exhaustion, not because it was pleasant? What if the area of the brain that they activated did not reward them with a sense of deep satisfaction, but only promised it? Maybe the rats excited themselves, since the brain told them that it was left to press only once-and something durable will happen? " - and quite reasonably, but, in our opinion, unnecessarily categorically replies to this:“ Olds and Milner did not open the center of pleasure, but what neurobiologists now call the reinforcement system. The area that they stimulated was part of the most primitive motivational brain structure that arose to encourage us to action and consumption . ”
В то время как Макгонигал отождествляет системы подкрепления и побуждения и отказывается замечать удовольствие в предвкушении — « когда возбуждалась система подкрепления, [испытуемые] переживали предвкушение, а не удовольствие », — Рой Вайз задается вопросом о том, существует ли четкая грань между предвкушением вознаграждения и получением вознаграждения 9 . Он считает, что ответ на вопрос не так очевиден, как поначалу может показаться. Например, если речь идет о пище: когда мы получаем вознаграждение — когда уже ничто не мешает нам пойти на обеденный перерыв, когда думаем об ожидающем нас за открывающейся дверью кафе обеде, когда ощущаем аппетитный запах, когда уже видим аппетитные блюда, когда берем в руки столовые приборы или только когда наконец ощущаем вкус пищи? Оказывается, что и вкус — лишь обещание награды в строгом смысле. Здравый смысл отказывается верить в это. Но экспериментальные данные показывают, что ради получения сахарина — сладкого, как известно, но лишенного пищевой ценности — готовы работать далеко не все крысы, а только те, которые ранее уже пробовали питательные сладости, например материнское молоко 10 .
И какова же здесь роль дофамина? Об этом — в одном любопытном исследовании. В нем показано 11 , что мутантные мыши, у которых полностью отсутствуют вкусовые рецепторы к сладкому (а значит, они не ощущают вкуса веществ, которые воспринимаются нами как сладкие), все-таки обучаются различать поилки с сахаром и без него и предпочитают пить из тех, что содержат сахар, и это сопровождается выделением дофамина. А будет ли выделяться дофамин, если мы заменим сахар на некалорийный сахарин? Нет, подсластитель не вызывает повышения уровня дофамина у мутантных мышей. И эти мыши не предпочитают раствор сахарина воде.
А что же у мышей без спровоцированного генетического дефекта, у нормальных мышей? У них в ответ на вкус сахарина дофамин выделяется, для них сладкое — это сигнал полезности, ведь они этому научились еще тогда, когда питались материнским молоком. Эти результаты указывают на участие дофаминовой системы во внутренних, сокрытых от нас метаболических аспектах подкрепления и в очередной раз ставят нас в тупик перед сложностью самого явления «подкрепление».
Но мы еще не рассмотрели сквозь призму новых методов последнюю из важнейших гипотез о ключевой роли дофамина в организации движения. Вы, наверное, знаете историю про двух лягушек, оказавшихся то ли в горшке со сметаной, то ли в крынке с молоком (доподлинно неизвестно, как они там оказались). Так вот, выбраться было совершенно невозможно, и одна из лягушек, потеряв надежду, вскоре утонула. Другая же никак не хотела сдаваться и продолжала энергично барахтаться в молоке. Неожиданно для себя она сбила масло, и это помогло ей выбраться.
Соблюдая все конвенции об этическом отношении к животным, некоторые исследователи помещают мышей, подобно этим лягушкам, в цилиндрический сосуд с водой, подобный узкому стакану, из которого нельзя выбраться (делают это они не только из любви к истине, но и чтобы найти подход к лечению такого тяжелого недуга, как депрессия). Мыши, как и лягушки, могут либо неподвижно «висеть» в этих сосудах, постепенно погружаясь всё глубже, либо совершать плавательные движения. От чего же зависит, сдастся мышь или продолжит борьбу? Выяснилось, что побуждает животное не сдаваться именно дофамин: оптогенетическая стимуляция дофаминергических нейронов VTA существенно увеличивала число плавательных движений в единицу времени 12 . Можно допустить, что данный эффект опосредован главным образом изменением мотивационного состояния, а не прямыми влияниями на механизмы организации движения. Однако в недавнем исследовании, опять-таки с применением оптогенетического метода, продемонстрировано, что дофаминовые нейроны черного вещества играют ведущую роль в контроле кинематических характеристик (например, скорости и ускорения) произвольного движения 13 . И это уже указывает на более тесную связь среднемозгового дофамина с движением.
Итак, активность дофаминовых нейронов связана с движением, но как? Обычно, когда существует связь, всех начинают интересовать подробности этой связи. И в 2016 году были опубликованы результаты двух работ разных коллективов исследователей, в каждой из которых мышь с закрепленной головой помещали на поверхность барабана, который может вращаться, если животное пытается по нему шагать.
Что это дает исследователям? В обоих случаях ученым хотелось иметь дело с бодрствующими подвижными животными, но одна группа пыталась при этом заглянуть в мозг при помощи микроскопа, чтобы увидеть работающие и при этом светящиеся окончания дофаминовых нейронов (если вы вдруг подумали, что для работающих клеток светиться — это вполне естественно, то огорчу вас: это не так, это очередное методическое чудо), а другая пометила зарегистрированные клетки особым веществом (нейробиотином), чтобы потом разобраться, были ли эти клетки дофаминергическими. И то и другое очень трудно сделать на свободноподвижных животных, здесь и оказался полезен барабан: ты вроде бы передвигаешься и даже бежишь иногда, но при этом остаешься на месте (подобно Алисе в Зазеркалье), а главное — голова хорошо закреплена и неподвижна, что равно необходимо для обоих методов. Самое интересное, что результаты этих работ оказались практически противоположны друг другу: начало движения сопровождалось кратковременной активацией дофаминергической системы мыши в одной работе и, наоборот, короткой паузой в дофаминовой активности — в другой 14 . Хотя вполне может оказаться, что всё дело в нюансах, например в немного разных временах — реакции могли начинаться чуть раньше или чуть позже начала движения, — в разных методах измерения и оценки активности нейронов, в присутствии вознаграждающих стимулов в экспериментальных условиях у одних исследователей и в отсутствии таковых у других и т. д. Главное для нас — то, что и эта гипотеза о ключевой роли дофамина в организации движения сохраняет свою актуальность и в очередной раз требует уточнений.
Итак, для всех основных конкурирующих гипотез о специфике участия дофамина в поведении найдены новые весомые доказательства; проблема только в том, что гипотезы по-прежнему хотя бы отчасти существенно противоречат друг другу. Совсем не хочется иронизировать по этому поводу, так как я убеждена: корень проблемы — в объекте исследования, тайна его глубока, его «секрет разгадке жизни равносилен».
Наталья Ивлиева, нейрофизиолог
1 Ex oriente lux (С Востока свет) — латинское крылатое выражение.
2 Ивлиева Н. Дофамин и счастье // ТрВ-Наука № 351 от 19 апреля 2022 года. trv-science.ru/2022/04/dopamine-i-schastje/
Ивлиева Н. Догадка о дофамине и сознании // ТрВ-Наука № 353 от 17 мая 2022 года. trv-science.ru/2022/05/dogadka-o-dopamine-i-soznanii/
Ивлиева Н. На путях свободы // ТрВ-Наука № 354 за 31 мая 2022 года. trv-science.ru/2022/05/na-putyax-svobody/
3 Tsai HC, Zhang F., Adamantidis A., Stuber GD, Bonci A, de Lecea L., Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning // Science. 22 May 2009; 324(5930): 1080–4.
4 Witten IB, Steinberg EE, Lee SY, Davidson TJ, Zalocusky KA, Brodsky M., Yizhar O., Cho SL, Gong S., Ramakrishnan C., Stuber GD, Tye KM, Janak PH, Deisseroth K. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement // Neuron. 2011. 72(5):721–33; Rossi MA, Sukharnikova T., Hayrapetyan VY, Yang L., Yin HH Operant self-stimulation of dopamine neurons in the substantia nigra // PLoS One. 2013. 8(6): e65799.
5 Милнер П. Физиологическая психология (1973).
6 Steinberg EE, Keiflin R., Boivin JR, Witten IB, Deisseroth K., Janak PH A causal link between prediction errors, dopamine neurons and learning // Nature Neuroscience. Jul 2013; 16(7): 966–73.
7 Chang CY, Esber GR, Marrero-Garcia Y., Yau HJ, Bonci A., Schoenbaum G. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors // Nature Neuroscience. 2016. 19(1):111–6.
8 Ilango A., Kesner AJ, Broker CJ, Wang DV, Ikemoto S. Phasic excitation of ventral tegmental dopamine neurons potentiates the initiation of conditioned approach behavior: parametric and reinforcement-schedule analyses // Front. Behav. Neurosci. 2014. 8:155.
9 Wise RA Brain reward circuitry: insights from unsensed incentives // Neuron. 2002. 36(2): 229–40.
10 Johanson IB, and Hall WG Appetitive learning in 1-day- old rat pups // Science. 1979. 205, 419–421.
11 De Araujo I., Oliveira-Maia A., Sotnikova T., Gainetdinov R., Caron M., Nicolelis M., Simon S. Food reward in the absence of taste receptor signaling // Neuron. 2008. 57(6): 930–941.
12 Tye KM, Mirzabekov JJ, Warden MR, Ferenczi EA, Tsai HC, Finkelstein J., Kim SY, Adhikari A., Thompson KR, Andalman AS, Gunaydin LA, Witten IB, Deisseroth K. Dopamine neurons modulate neural encoding and expression of depression-related behaviour // Nature. 24 Jan 2013; 493(7433): 537–41.
13 Barter J., Li S., Lu D., Bartholomew RA, Rossi MA, Shoemaker CT, Salas-Meza D., Gaidis E., Yin HH Beyond reward prediction errors: the role of dopamine in movement kinematics // Front. Integr. Neurosci. 9:39 (2015).
14 Howe MW, Dombeck DA Rapid signalling in distinct dopaminergic axons during locomotion and reward // Nature. 2016. 535 (7613): 505–10;
Dodson PD, Dreyer JK, Jennings KA, Syed EC, Wade-Martins R., Cragg SJ, Bolam JP, Magill PJ Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism // Proc. Natl. Acad. SCI. US A. 2016. 113 (15): E2180–8.