Translated material
Attempting self -organization
By the decade of the project "Corps of experts in natural sciences"
There is no question of the anniversary, but the date is round, and this is an occasion for some generalizations. In October 2007, the MCNMO was launched by the initiative project “Expert Corps” (KE) [1]. These 10 years have contained a lot of samples and errors, losses and finds, mountains of technical work and many substantial examinations. At the output - a large current database and experience that we would like to share in this text with colleagues. There is no way to describe this whole story here from its very beginning, but by links to the pages of the site [1] you can fill in the gaps.
Features of the "snow coma"
When planning the first polls of the KE, in which the quoted specialists from the “Stern lists” [2] asked to recommend experts, that is, to name qualified and respected colleagues of the same or related specialty, we hoped for several turns. They thought that those who would be called n times would become recommenders at the next round regardless of citation.
However, analogies with the “snow lump” in sociological surveys did not work: apparently, in specialized polls, the circle of people actively working in a particular area often turned out to be too narrow. And to some communities in which typical citation was low, and therefore there were few recommenders - “com” simply did not work, surveys of neighboring communities did not help very much.
There were more and less successful attempts to reduce the quotation threshold as a criterion for recommenders [3]. It was repeatedly discussed how to choose the number N for different disciplinary communities. However, when expert work began directly, it became clear that expert qualities for any N require direct experimental verification. The role of polls in the project was, however, very great and did not come down only to obtaining recommendations: the participants in the surveys helped to expand the lists of citation and, most importantly, in clarifying the specializations of colleagues. In the past two years, a “big poll” was held [4] in all specialties at the same time, and new names continued to appear, but among them people of the youngest generation had already prevailed.
3199 people responded to the kae profiles, of which 82 considered it necessary to declare the refusal of participation. The rest (32.6% of the total number of people in the quoting lists as of October 2017) either filled out the forms of the entire recommendations (2453 people), or-only in terms of information about themselves (664 people). Thank you so much. Apparently, the questionnaires did not reach some addressees due to problems with e-mail.
During the surveys, 9709 people received recommendations (at least two recommendations - 3150, at least three - 1665, at least five - 733). Powerfuls like “The quoted people will choose themselves” turned out to be completely unnecessary: half of the people included in the quotation lists were not recommended, and among the ones recommended at least once, they includes a little less than half. The rest of the recommended - 2660 foreign colleagues, as well as 2293 Russian -speaking specialists, the citation of which is below the threshold in the lists [2] (these are, as a rule, representatives of the inexperienced directions).
( Not ?) The fullness of the list of recommenders
This is a kind of resource: all academicians and members of the RAM (including the former RAMS), participants in large experimental collaborations, former dynasties scholarships and other similar groups of young scientific people, as well as colleagues who were involved in examinations on “secondary” recommendations (see below). Data on their publications are updated from time to time, and some names gradually replenish lists [2]. In parallel, the traditional declarative mechanism operates (works both by mail and through the site). And finally, Sergey Krashakov is excavating in the RSCI (this is almost a complete, albeit very inaccurate list of all scientific and sub -scientists of the Russian Federation).
Thanks to the search for these independent channels in the quoting lists, now almost 7 thousand people working in whole or in part in the Russian Federation (72% of the total number in these lists). It seems that the list of Russian recommenders contains a significant share of actively working specialists with noticeable citation, and in this sense goes to saturation.
On the contrary, there is no need to talk about the fullness of lists in the diaspora, it is unlikely to be achievable - and it is not obvious, which is needed. Representatives of the diaspora mainly appear in the database (and in the case of criteria - in the quoting lists) according to the recommendations of colleagues or (much less often) they themselves ask to check their citation and include in the lists. These are mainly people who maintain contacts with colleagues in the Russian Federation.
Actually examination
Examination customers over the past years were different organizations, and the requests themselves were different. For example, RVC asked to select experts for projects, and carried out the evaluation process independently. The Troika-Dialog company, on the contrary, turned to organize a full cycle of project evaluation, Rusnano-for the examination of the already supported educational projects and the analysis of the “nanotechnological” education as a whole, Skoltech-for the internal competition of joint projects with MIT and for a scientific audit of units. The most massive expert application of KE were the youth contests of Moscow State University (larger competitions in this institution were held somehow differently). For examination at Moscow State University, a conjugated base was created, and once a year between the bases of Moscow State University and Ke there was an exchange of information.
We discuss here only impressions of examinations that observed at all their stages. In such cases, specialists have always been repeatedly recommended in such cases by specialists, their role is most responsible and decisive. The coordinators, in addition to access to the KE database (or the conjugated base of Moscow State University), had the right to choose experts, if necessary, not only from these bases - which is inevitable in case of directions with typically low citation.
In real practice, especially for narrow and low -cycle areas, they were guided by the threshold of n = 3 (at least three recommendations) or implemented the recommendatory principle “according to the chain”. The related specialists of the related profile were asked: who would have been an expert on such and such a question? - And most often they recognized the names of people who should be asked more precisely. For all participants in the examinations, an expert story is accumulated in the bases (in particular, a radical difference between the assessment and estimates of other experts who solved the same task is noted). In total, to date, 1096 people have been invited to participate in examinations at least once (among them were refused and unable), 391 have been repeatedly invited and participated - 391.
With experts and coordinators, the interaction has always been much simpler than with customers. To begin with, the most progressive representatives of customers, saying that they need an independent examination, usually cannot immediately firmly formulate what specific questions they want to get answers from experts. If projects, then still half -worn - just want to know which support (but criteria usually indicate vaguely). But if the audit is quite difficult to understand what to evaluate. During the discussions with each customer who “wanted good”, regulations were gradually born, the inalienable parts of which, of course, were the requirements for the materials provided for evaluating. And then a narrow place arose: few of the customers were able to provide all information in the format, which he himself agreed ...
The most important feature of the expert work of KE is the choice of expert specialization, as close to the specialization of the evaluation object. This is especially important for the so -called “multidisciplinary” cases, because there are no “multidisciplinary” experts, and the more “multi” the object - the more people from different disciplines should evaluate it.
The classifier, which we consider to be central in the entire structure [6], is a way of preliminary sorting by disciplines and specializations; Keywords are a useful addition to the classifier, and, in addition, the coordinator can determine the specialization with maximum accuracy using relevant information about the publications of each person in the KE database.
It is not a matter of experts to make organizational and financial decisions based on the results of the examination, but, of course, we were interested in the final result. There were few examples when the conclusions of the examination were taken into account by customers completely or to a large extent; More often the result was the opposite. Nevertheless, we will continue to conduct examinations and accumulate an expert story. Only she helps to finally understand, "who is who."
Foreign policy
It’s not that the Ministry of Education and Science and (later) Fano did not notice us all these years ... We were very noticed, and all sorts of signals came from there. Most often, the signals came from individual lower employees as from individuals, in the style of “give a list of experts to evaluate megagrators”-of course, without any explanation of how this assessment will be carried out, and without any information about the subject (“in physics in general”).
In such cases, we sometimes called the most reliable and wide in horizons of experts, about which there was already confidence that they would definitely not be reviewed outside their specialization, but will be able to name the necessary specialists. Participation in the organization of competitions in such cases was firmly avoided: through the state stations, carrying out the correct regulations of the examination without distortion - it is like to run into a tank column on a scooter. And we still tried to save our scooter to create alternative precedents.
Well, in fact, what can be the intersections between KE and the Ministry of Education and Science? We are in different planes. The Ministry of Education and Science collects information about all institutions and researchers, and KE - about the most active, noticeable in publications and reviews of colleagues. The Ministry of Education and Science studies the collected information at the state statistics level, and KE at the level of scientific specializations. The Ministry of Education and Science creates the “Fedyukin map of science”, and Ke “ -“ A map of minerals in the territory of Russian science ”[7]. Something like in a joke: "I do not give a loan, and the bank does not trade with seeds."
Speaking seriously, we are completely sure of the need to analyze the Russian scientific situation at a purely professional level, regardless of state -owned stations that are not able to produce such an analysis: what specific areas are still competitive, what specializations and disciplines support a solid average level and still ensure the continuity of generations, what and where there is a human resource for teaching modern special courses in universities, etc. It cannot be completely excluded that in some directions such an analysis can be supported by the non-state industry and companies. We are ready to discuss the initiatives of this kind and are sure that the technical tools created in 10 years can significantly help in preparing information for such an analysis.
In 2014–2016, our interaction with state stations happened in the order of supporting the efforts of the Commission on Public Control in Science ( Rascommission.ru ). The case was connected with the organization of the assessment of the institutions conducted by the Fano. A noticeable number of members of the commission was included in the Fano working group for the development of documents by which it was planned to conduct this assessment. Of course, it was a story from the series “As always”: a detailed and self-confined regulation was shocked in the authorities, the “collection of information” is still taking place in a standard Minobrovsky regime, the terms of evaluation are again firefighters.
But one of the preserved elements seems important to us even in the unfavorable situation that has developed now. We fought the whole world for a large number of specialized reference groups in order to minimize the “comparison of the long with green”, and this was largely achieved (alternative options that initially came from the Ministry of Education and Science and RAS were both unusable, but very differently; in this place, we suddenly successfully succeeded in the role of centrists).
Plans and prospects
You can write detailed materials with a variety of statistical information about each of the aspects mentioned above, not to mention the reasons that give such materials for analytical work. Whether we have enough strength to do all this, depends on many circumstances, and above all, on the appearance of ready -made volunteers from younger and middle scientific generations.
We will do everything possible to maintain current lists [2], but it should be understood that their existence is directly related to paying for the contract with Thomson Reuters and the manual labor of operators. In the absence of grant support for the KE project (it has been not there for four years), this is possible only at the expense of contracts for additional work, the implementation of which requires, in turn, the wages of technical employees and very significant labor costs of members of the working group, traditionally operating on a public basis.
We are ready to organize expert support for qualified customers who can formulate assessment tasks. And recently we think that in conditions of unexpected heyday of scientific popularization, KE could significantly help reporters avoid obviously inaccurate and erroneous information. The KE project is open to proposals and cooperation with people who share our ideas about the need to create precedents of normal intracledic life even in the conditions of an inadequate state scientific policy.
ACKNOWLEDGEMENTS
The project “Corps of Experts” at different stages had and have wonderful employees. We are deeply grateful for the long -suffering and huge labor costs of Vladislav Sabraui, Olga Garmash, Alexander Prokofiev, Ekaterina Malinkina, Yulia Vakhteeva, Inna Krashakova and Grigory Kopelevich. We are grateful to our many consultants (see [1]) and separately to Sergey Krashakov, who helps the project in daily mode.
KE stability for six years gave the support of the Dynasty Foundation.
The decisive role in the initiation and development of the KE project, as well as in its protection from the conjunctural well -wishers, was played by Vitaly Arnold (1968–2017).
Mikhail Gelfand, Mikhail Feigelman, Galina Zirlin
1. Project website: www.expertcorps.ru
2. Quoting lists: www.expertcorps.ru/science/whoiswho (criteria are described at the beginning of the page).
3. Polling methods: www.expertcorps.ru/science/methods
4. M. Gelfand, M. Feigelman, G. Cyrlin. Big survey // TRV-hunger, No. 188 of September 22, 2015. S.3.
5. Memorial list: www.expertCorps.ru/science/whoiswho/memlist
6. Unified scientific rubricator: www.rascommission.ru/rubricator
7. Maps of minerals - Search on the territory of Russian science: www.expertcorps.ru/static/cms/map_final.pdf