Expertise: their and our

What do officials say and do to establish a scientific examination in Russia, and what should it really be? The article by Pavel Chebotarev , a doct is devoted to these and other issues. physical. Sciences, G.N.S. Institute of Management Problems named after V.A. Trapeznikova RAS, member of the Society of Scientists.

Starting from the “Black Thursday” on June 27, 2013, when the government bill developed in the strictest secret, the scientific community is trying to prevent the implementation of a catastrophic scenario, including the sale of real estate of the Russian Academy of Sciences, subordination of scientists to officials, the closure of many institutions and the complete destruction of the conditions for scientific work in Russia.

Six months have passed. The law is signed in the form close to the initial: the institutes are torn from the wounds and re -incidentally reassigned to the "agency" - Fano. Does this mean a final defeat?

On December 20, on the day of the security officer, having signed the mercy of Mikhail Khodorkovsky in the morning, Vladimir Putin holds a meeting of the Council on Science and Education, dedicated to "steps on the further development of fundamental research in Russia." He calls Fano’s non -interference in scientific activity (and its leadership), confirms an agreement on an annual moratorium on property and personnel changes in the Russian Academy of Sciences (though it now mentions only property) and emphasizes twice that even the property of the Academy, which is not needed today for science. “Because then in the interests of science it will not be obtained [this] when there is a need” [1].

However, the call for accuracy in handling property does not oblige nothing to anything, especially when it is said: “Something, probably, can be conveyed, something to realize.” You can’t get the promises of the moratorium either. Vladimir Fortov fairly qualifies them as “oral opinion” and asks to reinforce it with an official assignment to the government on the practical implementation of the moratorium. In addition, he emphasizes the need for “clear records” that exclude the intervention of Fano into the sphere of research. He remembers the disappointment after the September assurances of the president about the consent with all the wounds proposed by the Russian Academy of Sciences, except for one secondary.

At the time of writing, there is no order to the government, about which Fortov asked, nor the final document of the Council meeting. And in general, there is no evidence of the power of power from a rude destructive scenario. Loiling speeches should not relax, but aggravate attention to the simultaneous practical actions of officials. At the same time, these speeches make the conversation not about a scientific catastrophe, but about effectiveness and effectiveness. Thus, the problem of an adequate assessment of scientific achievements comes to the fore.

Expertise hymns at a meeting in the Kremlin

If you look at this problem through the prism of the opposition “numerical indicators -experts”, then it catches the eye (and this may seem unexpected) that the enthusiasts of bibliometry and formal performance indicators at a meeting of the Council were not heard at all.

The fact that “the system [assessment of the effectiveness of institutions] should be based on expert assessments as much as possible,” Dmitry Livanov says (his words are given in the quotes), Vladimir Fortov, Mikhail Kovalchuk, Andrei Fursenko, Evgeny Primakov and other speakers. “No scientific indicators can replace the opinion of a specialist” (Fursenko), “Scientific is an additional indicator of the Hamburg account for deep expert estimates” (Kovalchuk), “We need to introduce expert criteria for assessing scientific work” (Fort), etc.

The reasons why the participants distance from the “general hobby” by science are different, but the Gudhart Law, which states that the numerical indicator loses its properties and destroys natural motives of activity as soon as it becomes a target indicator, has already acquired the status of Common Knowledge.

Expressing this general trend, A. Fursenko proclaims: “I propose to entrust the government together with the Russian Academy of Sciences to develop and approve the work plan on expert scientific support for all significant programs and projects not only in the field of fundamental research, but also on other activities that determine the socio-economic development of Russia.”

Here there is a feeling of deja vu.

Deja vu

In the mid-90s, they talked a lot about increasing the role of examination in public administration. The need for the adoption of the Federal Law on Expertise and Expert Activities was realized. The draft laws were prepared, but the law was not adopted [2].

Dmitry Schmermling and Anna Weinmaker, in the yet not published article “The current state of expert activities and expert assessments in public administration” recall that in November 2001, a civilian forum was held at the Kremlin Palace of Congresses, where one of the central issues was about the state, civil and public examination [3]. And it is not surprising that among the instructions, which, according to the results of the forum, gave the recently elected President of the Russian Federation V. Putin, again was the development of the Law “On Expertise and Expert Activities in the Russian Federation”. The draft law was again developed, but buried by the Ministry of Justice of Russia [4]. Since then, the general law on expert activities, the statements of which are heard regularly [5] (see also [6-9]), and not. What interferes with its acceptance? Why do insurmountable administrative barriers get up in his way?

Official and examination: love and fears

A plausible explanation is that officials need an examination as a beautiful slogan, and not as a mechanism with unpredictable results. Public regulation of examination procedures will inevitably lead to their greater transparency, which means to poor handling.

In the field of assessment of scientific activity, this pattern is manifested quite clearly. The statements of D. Livanov and A. Fursen-ko on the dominant role of professional examination were already given above. But when moving from ceremonial speeches to everyday practice, the picture changes. According to the rules for evaluating and monitoring the performance approved by the government on November 1, 2013 [9], scientific organizations are evaluated according to formal criteria, and the examination is provided only for the “analysis” of numerical data presented by organizations and the activities of the units of the institutions that are threatened with liquidation.

Simply putting, the rules prescribe experts to study the columns of numbers, and not what the employees of the institutes were made. This approach was confirmed at a round table on assessing scientific organizations held at the Ministry of Education and Science on December 5, 2013: according to eyewitnesses [11], “The proposal to evaluate the expert commission not only“ outsiders ”, but also leaders” did not cause the developers enthusiasm there.

And here is what is significant: in the rules, the examination is not only narrowed to the limit, but in general it is only mentioned in passing - not a word is said about its regulation (as, by the way, about the method of packing fifty numerical indicators). Indeed, according to the logic of officials, it is necessary to “regulate” the institutions so that they disciplinedly collect the data (and the corresponding order is already written [12]), but why deprive themselves of the space of maneuver when making decisions?

Due to this state of affairs, it is hardly worth expecting that the glory about the professional examination at the December Presidential Council will not today lead to rewriting the rules of evaluation. D. Livanov’s “double -mindedness” in this matter (ceremonial speeches is one, practice - another) is absolutely characteristic, if not “reference”. It is even more eloquent that the experienced apparatchik A. Fursenko in the above “order” proposes to develop not the law on examination, but the “work plan on expert scientific support” of all significant programs and projects that determine the socio-economic development of Russia. In other words, there is a desire to plan work and allocate funds. And there is no desire to regulate anything.

What is an examination?

The examination must be determined extremely specifically, says Nina Belyaeva, one of the developers of the draft Law on Expertise of 2002, head. Department of Public Policy of the Higher School of Economics [4]. This is the only way to separate it from related activities requiring other legal regulation.

“Examination is the establishment of the correspondence of the proposed sample to the regulatory model using the procedure that is approved by the regulatory act” - this is its sufficient definition. It recognizes the examination far from every evaluative judgment obtained from a professional, but only a verification of compliance with the standard completed according to the approved rules. If there are no standards, then they must be developed, and this is an important part of expert work (which must be distinguished from the examination in the narrow sense). However, this work involves a certain metaxpertise, because The standards system must also satisfy certain standards. So, again, we quote N. Belyaev: "A normal examination is very strictly regulated procedurally."

We cannot wait for mercies from ...

In this sense, what officials love is not an examination at all.

Their love for "examination" has two obvious reasons: the slogan about it sounds great, and it itself is easily profanated. If you make an examination professional, transparent and strictly regulated, then half of the charm will disappear immediately. It means that no matter what the authorities say, wait for them to “make us” an examination that we will like, do not have to. And a profanated examination, perhaps, is worse than the law of Gudhart and other vices of bibliometry. Therefore, the choice in favor of the examination is meaningful only when there are guarantees for ensuring its quality.

Note that the question of who and how organizes the examination is so fundamental that he caused a sharp polemic atypical for such events at a meeting in the Kremlin. V. Fortov insists on the expert monopoly of the Academy. It recalls: “The Law on Reform and the Regulation on Fano establishes that it is the Russian Academy of Sciences that monitors and evaluate the effectiveness of scientific organizations, regardless of their departmental affiliation” [1]. But, he notes further, in contradiction with the law, the government decree of November 1, 2013 assigns these functions to the Ministry of Education and Science of Russia.

D. Livanov sarcastically rates: “We know the experience of assessing the scientific organizations of the Academy of Sciences itself, when it also included scientific organizations: 80% of organizations were attributed to those that work at the world level. We know perfectly that ... this is far from the case. " In his version, the main condition for the quality of the examination is its independence.

But the question “by which forces to conduct an examination?”, Unlike the previous one, when discussing in such a meeting, it turns out to be ritual-ribroid. There are no discrepancies between opponents. D. Livanov: “The expert community for assessing scientific organizations will necessarily be formed with the involvement of leading Russian scientists actively working in a particular field of science.” Further, he emphasizes their “key role” in the examination. V. Fortov calls on to rely on expert criteria for assessing the scientific work developed by scientists themselves, "and to send tens of millions of rubles to real science, and not to empty paper -making."
So, the authorities and scientific generals sing hymns of professional examination, but do nothing exactly for its formation. As if inviting domestic scientists to self -organize and show their expert competence to the world. We remember from political science that the more acute the competition of two large parties (in this case, it is the Ministry of Education and Science and RAS - Livanov and Fort), the more the voice of citizens is heard and the more the third force can achieve, which did not have a serious influence.

Self -organization is needed

And what about the scientists themselves? Since 2007, the Enthusiasts forces have been developing the project “Corps of Experts on Natural Sciences” [13], which uses both expert and scientific approaches to form the expert community.

On December 30, 2013, the public control commission and the results of reforms in the field of science presents a draft regulation of a professional assessment of scientific organizations [14], based on the approach of the “Expert Corps”. The project was discussed from December 3 to 24; Participants in the discussion, which expressed a number of comments and proposals, mainly noted its high general quality. Other similar developments are unknown to us.

So far, very few people are being involved in this activity, and at the moment the contours of the regulations have been determined only for the natural sciences. Even for mathematics and computer science, most of the work has yet to be done, not to mention other disciplines. The most time -consuming part is the formation of the expert corps. In addition to the exact sciences, something has been done here in the field of linguistics.

If scientists do not present a well-developed project, after a year or two V. Fortov and D. Livanov will say something like the following: “Naus was a chance to fully participate in the assessment of scientific groups. You did not use it. We, arguing with each other, did it as we knew how. For you, the door was ajar, but you did not enter it. Now we have what we have. ”

Those who want to participate in this work can contact the developers through the website of the public control commission - www.rascommission.ru .

The self -organization of the expert community is needed, of course, not only for the assessment of institutions, but also to determine the promising areas of the development of science, to form the position of scientists on all significant issues. Without this, we are doomed to be hostages of other people's interests.

1. http://state.kremlin.ru/council/6/news/19865
2. http://parzefal.com/source/000382.pdf ; www.pressarchive.ru/ekonomika-i-jizn/1997/09/30/%20254893.html
3. D. Schmerling, A. Weinmaker. The current state of expert activities and expert assessments in public administration // XV April International Scientific Conference on the Problems of Economics and Society. M.: NIU HSE. In the press.
4. http://polit.ru/article/2002/12/16/611864
5. www.hse.ru/data/2012/10/08/1247168533/expertisaproektovnpa.doc
6. www.lawmix.ru/commlav/126
7. www.portalus.ru/modules/different/rus%20Readme.php?subaction=showfull&td=1271064962&ARCHIVE=&START_FROM=&UCAT=19&CATEGory=19
8. http://salon.extech.ru/salon8/html/result.shtml
9. www.sochi-pravo.ru/moi-stati.html
10. http://pravo.gov.ru:8080/page.aspx?67047
11. http://trv-science.ru/2013/12/24/i-vot-obshhestvennoe-mnene/
12. http://regulation.gov.ru/get.php?View%20ID=3&DOC_ID=23934
13. http://expertCorps.ru/
14. www.rascommission.ru/index.php/%20Documents/statements/63-otsenka