Wounded competition

The other day, on the website of the Ministry of Education and Science, the results of the competition of large projects of 2020 were published [ 1 ]. This was the first mass competition, significant on the country's scale, where the examination was conducted by the Russian RAS. These forces are rather big: in the advertising prospectus [ 2 ] it is indicated that 7265 people are part of the wound experts of the Russian Academy of Sciences, of which 32% are members of the Russian Academy of Sciences and foreign academies, 88% are doctors of sciences. Unfortunately, although many strong groups won quite deservedly, the organizational pancake came out lumpy: there are a number of complaints about the organization of the competition.

First of all, the technical base - sites where the applications were downloaded and where the review was held - was of poor quality. When loading, unexpectedly introduced by the competitive documentation (hereinafter referred to as CD) restrictions on the size of the substantive file files, and the text had to be urgently and significantly reduced - cut on the living. In case of refusal of examination, the expert did not have the opportunity to indicate the reason for the refusal. Sites are bulky and work slowly. It is strange that many years of experience in the work of the website of the RFFI and RNF competitions were not taken into account - there have never been such outrages there. I must say that the NTP Directorate (the Directorate of Scientific and Technical Programs), which is famous for the clumsy site and amazing bureaucratization of its contests [3] engaged in the support of the competition [ 3 ].

Further, it is worth noting that the examination procedure was opaque. The following is written in the CD:

“ 9.2.7 Expertise is carried out according to the criteria approved by the Expert Council, with the participation of the experts of the Federal State Budgetary Institution“ Russian Academy of Sciences ”selected by the experts and with the involvement of specialists (experts) in the relevant fields of sciences and (or) organizations with the necessary experience and qualifications for the examination of scientific projects. As a result of the examination, applications for participation in the competition are set by points for each criterion. ”

As you can see, there is no indication of how many experts review each application, in what cases and how many additional specialists are involved. Since the final points were only multiple of the whole or semi -percent numbers, it can be concluded that no more than two experts were reviewed each application, and the total score was displayed as the average arithmetic. For comparison, I note that during the examination of megagranes, two Russian and two foreign experts review each application and the head of the group of experts writes at their reviews. The latter is very important, since each expert is with its own character, subject to emotions and points set by him often differ from the assessment based on his review.

The conclusion of the head of the group of experts allows you to balance the assessments and to some extent replaces group consultations of experts (consensus sessions [ 4 ]). It should be noted that the cost of a megagrater is only 90 million rubles, and the cost of a large project is 300 million rubles, that is, the price of error for Russian science in this competition is much larger.

It is also surprising that the vast majority of applications received money to the maximum - 100 million per year. Is the St. Petersburg Institute of History of the Russian Academy of Sciences the project “History of a letter of European civilization” also need almost 300 million? Do not get rid of the sensation of the ill -conceived organization of reviewing.

And finally, the main flaw is the organization of the examination itself. As an expert at the Russian Academy of Sciences, in early July 2020, a letter received a letter signed by the President of the Russian Academy of Sciences Alexander Sergeyev with an invitation to take part in the examination. Of course, I agreed, but warned that a conflict of interest is possible, since I myself am a project presented to the competition on the same section: mechanics and mechanical engineering. Three days later, two letters were sent to me with the names of those projects and links to access the review system. The applications were not in my area (management processes), but according to the material science, but the system did not give the reason for the refusal to be reviewed, and I began to think about it. Who and how appointed experts if they gave me projects on material science, to which I have nothing to do? What would happen if I had not refused? If the expert, who himself participates in the competition, gives the project high grades, he acts against himself, that is, a conflict of interest is created. And obviously, no one took into account a possible conflict of interests. Moreover, it can easily imagine that a group of influential persons close to leadership has their own ideas about the level of one or another profile institution or university and these preferences determine the choice of experts and evaluation in advance. This cannot be called an impartial examination. It may well be that influential persons took and divided projects among themselves for reviewing, having agreed on points. If this is not so, how did the examination take place?

If there was a transition from a wide competition to the competition for the branches of the Russian Academy of Sciences, then this leads to an unleasable conflict of interests, since inside the department all strong organizations are known (usually they are presented in the department with directors of academics). Therefore, potentially strong projects in the scientific field of the department are known, and the examination becomes not external, but internal.

However, an objective competition of projects within the university or within a scientific organization, if there is no external examination, is impossible, because the desire to give more points to your, close or familiar project (“Well, how not to be happy about the native man!”). Therefore, if the competition is held inside a small community, where everyone knows each other and have the opportunity to negotiate, then this is no longer a competition, but a deed. In this case, it would seem, there was a chance to balance the decisions at the final stage, at the level of the competitive commission, considering all the results of the examination at once. But this was also impossible, since the competitive commission was engaged only in the calculation and comparison of the scored points. It would be better if, as in the competition of megaggrators, the reviewers were appointed a separate commission for the expert council, the decision, the reviewers were anonymous. Or if the reviewers were assigned by a computer, as the RNF began to make. But this is not visible in the documents.

Bottom line: Huge money for science was again distributed in a doubt way. And the participation of the Russian Academy of Sciences in the examination did not make it impeccable. Is it possible to somehow correct the situation, cure this competition very important for the country? I think yes. For this, in my opinion, it is necessary to implement several key decisions.

The main of them is to replace in the regulation on the competition the competition committee of the ministry with the competitive council (expert panel), consisting of world -found scientists, which should determine the winners of the competition based on the results of the examination. It is important that the composition of the Council satisfies the following conditions:

a) all members of the Council must have a degree and international recognized reputation;

b) members of the Council should be affiliated with various departments (universities and scientific organizations of various subordination);

c) the field of competence of the members of the Council should cover all areas of sciences in which the competition has been announced;

d) None of the members of the Council (except, perhaps, chairman of the Council) should occupy high administrative posts (director/ deputy director, rector/ vice, etc.).

It is the expert council, consisting of qualified scientists, should prepare the regulations on the competition, configure its parameters: quotas for different fields of sciences, permissible boundaries for the number of points, etc. He must also determine the composition of documentation in applications and reports, reducing the number of formal-bureaucratic information. This approach has justified itself in the contests of megagrators [ 5 ]. There is an expert council in the competition discussed, but he does not make decisions.

The requirement (d) is very important for a number of reasons. Firstly, scientific administrators are usually very busy and cannot devote enough time to the analysis of the results of the examination. Secondly, each administrator represents its organization and, on duty, should defend its interests, which creates a potential conflict of interests and risk of bias in the assessment of projects.

The importance of taking measures to exclude a potential conflict of interests is noted in the documents of the Council on Science, who worked under the Ministry of Education and Science in 2013-2018 under the leadership of Alexei Khokhlov, the current vice president of the Russian Academy of Sciences [ 6 ]. The importance of the factor of impartiality and independence from the organization is also indicated in the rules for the examination of the European Commission projects. In the memo, experts and members of the FP-7 program panels say [ 4 ]:

The Expert Works as an Independent Person. He/she is deemed to work in a passonal capacity and in Performing the Work does not represent Any Organization. (The expert works as an independent person. He/she is considered to work under personal responsibility and does not submit any organization when performing work.)

Unfortunately, in recent years, in many bodies of the collegial leadership of science in Russia, the majority of places began to occupy a “general” (for example, in the Council of the RFF [ 7 ], the Interdepartmental Commission to assess the effectiveness of scientific organizations [ 8 ], etc.).

Another important condition for an honest modern competition is the creation and launch of a convenient Internet system for filing and reviewing applications. The current tools coming from the NTP directorate are inconvenient and should be rejected. You can even at first borrow convenient and familiar to many RNF and RFFI systems.

I hope that the leaders of the aforementioned respected departments with understanding will react to emotional criticism in a note written, as they say, in hot pursuit. I would like to hope that during the next competition, the level of organization will correspond to the level of financing of the winners. By the way, do you need grants of 300 million rubles.? Will it become much less scientific results if you give the winners “only” for 150 million? But in this case, two billion per year will be released, due to which you can save the contest “A” of the RFFIF and reassure the scientific community ...

Alexander Fradkov,
Head. Laboratory IPMASH RAS, co -chair of the Council of Onr

↑

"The role of the Academy of Sciences was only in the transfer of a list of potential experts"

The Academy of Sciences does not confirm the thesis that the examination of the competition was conducted by the Academy of Sciences. Representatives of the Russian Academy of Sciences emphasize that the competition was held by the Ministry of Higher Education and Science in accordance with the Decree of the Government of the Russian Federation of December 27, 2019 No. 1902. Its paragraph suggests that the second stage (substantive examination) is carried out by members of the expert council of the Ministry with the participation of experts of the Russian Academy of Sciences selected by the Council. For this, the Academy of Sciences conducted a special survey of the scientific community, as a result of which a preliminary list of 3,500 people was received. Of these, 1883 people were new, who had never participated in the examination of RAS projects. After the selection of candidates in science and other criteria in the Ministry of Education and Science, a list of experts in the amount of just over 1000 people was sent.

The candidates were selected specifically for this competition. The Academy of Sciences provided their list to the expert council of the Ministry of Education and Science of the Ministry of Education and Science, in order for members of the Council, when determining who will conduct an examination of a particular application, they could attract at their discretion certain experts who proposed the Russian Academy of Sciences. On this, the role of the Russian Academy of Sciences in the implementation of the examination has ended.

All expert actions were carried out by members of the Council, and the information and technical support was carried out by the NTP Directorate. It was precisely, inside her electronic system, she appointed the examination to those experts who were appointed by this or that member of the council.

The Russian Academy of Sciences, its structural units, branches of the Russian Academy of Sciences were not involved and did not participate in the examination either on the basis of the regulatory framework or in fact. Members of the Council are leading scientists of the countries, some of them have the status of academics, but they acted in personal quality, and not as representatives of the Russian Academy of Sciences.

The independence of the competition from the Russian Academy of Sciences may be indicated by the fact that the grants have not received many leading academic institutions. For the first time, the ministry checked more than 700 applications for plagiarism and self -plagiat; Just due to the presence of the latter, some leading universities and institutions of the Russian Academy of Sciences did not receive grants. For the examination, 300 applications were selected. The shortcomings that were noted in the article by A. L. Fradkova, my interlocutors associate with a lack of time for holding a competition, inability to spend it in the usual conditions due to pandemia and some logistics errors made by the organizers in the conditions of a strict time deficit in the midst of the vacation season. The ministry sought to have time to complete the competition until the end of the budget year. Nevertheless, colleagues believe that the strongest actually won. Although there are many dissatisfied among the losers. Of these, the academic part believes that if the examination in the competition conducted RAS, they would be honored winners. The university part believes that the Academy of Sciences pulled the blanket over itself.

N. D.