Translated material
How Rosobrnadzor works
In January 2018, employees of the Center for Institutional Analysis of Science and Education of the European University in St. Petersburg prepared a report with the results of an analysis of open data on control and supervisory activities in higher education 1 . TrV-Nauka publishes in an abbreviated form those results related to the work of experts participating in inspections of universities.
Rosobrnadzor is an agency in which the department for monitoring higher education has no more than fifty people. Without experts from the university environment, officials would not be able to conduct hundreds of inspections a year. According to the registers, Rosobrnadzor can contact 2,647 experts. 1
Who do we expect to see as an expert? Firstly, experts representing leading universities, and therefore familiar with the “best practices” of the Russian higher education system. Secondly, it is logical for the expert to be involved in the core activities of the university - teaching and research - and demonstrate competence in it. Finally, the expert should not be seen to engage in ethically questionable practices. Can Rosobrnadzor experts be called experts according to these criteria?
We examined a list of almost 800 experts accredited to participate in control and supervision activities. Most of these people work in universities. It cannot be said that strong universities are prominently represented on the list: only 11% of experts work in strong universities. Of the 45 leading universities, only 21 are represented by at least one expert, and only three universities account for half of all experts from strong universities. Moreover, on the list you can find teachers who work in ineffective universities. According to performance monitoring data, 15% of experts represent universities that did not receive effective status in 2015. Comparing various monitoring indicators, we can conclude that the majority of experts work in universities, which are neither the best nor the worst. In other words, the expert’s university is no different from the average Russian university. However, the indicators of expert universities differ greatly from the indicators of leading universities. Universities have especially low indicators of experts in publications in the Scopus database - accredited experts came mainly from those universities that either do not know what international science is, or do not know how or do not have time to publish.
It is believed that experts are representatives of the university environment who are themselves involved in teaching and research, and therefore will be able to assess the situation in a particular university. However, experts whose activities are primarily related to teaching, rather than management and preparation of documentation, are rare. We found that 75% occupy a leadership position; the educational and methodological department is prominently represented among the university departments - 35% of experts in their universities are engaged in educational and methodological activities. Since Rosobrnadzor’s inspections are focused on documentary reality, universities have established its efficient production: somewhere there are entire departments working to create such documentation. Apparently, representatives of methodological departments come to check how well the paper reality is built in other universities, and not to understand the essence of the matter.
We did not have the resources to check the integrity of all Rosobrnadzor experts, but we found enough examples to begin to seriously doubt the agency’s ability to conduct high-quality selection. Experts are found in the Dissernet database: 13% of experts (of the number of protected experts) in the role of dissertation author, supervisor or opponent. This figure is the result of a relatively random expert coming into the field of view of Dissernet, since Dissernet has never set as its special task the verification of experts 2 . Borrowings were found in 31% of experts' dissertations. Realizing the importance of moral character, Rosobrnadzor calls on experts to follow the code of ethics. However, the service does not in any way protect from participation in the examination those who have already been found to have violated norms that are directly related to scientific and educational activities. Behind their signature you can find negative conclusions that can paralyze the activities of universities such as Shaninka (expert Svetlana Aleksandrovna Nikonova 3 ).
Moreover, experts found to have violated academic ethics not only continue to inspect universities, but are also shock workers involved in dozens of inspections per year. Among the most active experts are M. Zaostrovtseva and L. Solomina. Previously, they worked at a private university - the Capital Academy of Finance and Humanities. In 2016, the academy was recognized as ineffective; in 2017, the university did not take part in monitoring - inspections from Rosobrnadzor were expected. There is virtually no information about these experts as teachers and scientists, with the exception of leadership positions in academic work and defended dissertations. At the same time, dissertations are full of massive text borrowings. Dissernet data show that L. A. Solomina’s dissertation for the degree of candidate of pedagogical sciences contains a lot of incorrect borrowings, primarily from M. Zaostrovtseva’s candidate dissertation. However, in Zaostrovtseva’s own dissertation, judging by the data from the same Dissernet, the percentage of incorrect borrowings is also very high.
Having examined the publications of experts, we can see among them a clear desire to increase the number of publications in a relatively easy and simple way, bypassing the standard procedures in high-quality scientific journals. Among the experts, there are twice as many of those who have published five or more publications in publications not indexed by the RSCI - 20% of experts versus 10% of non-expert teachers. We encounter “junk” Scopus, co-authored articles that raise suspicions, and citation cartels. It is enough to spend five minutes studying the profile of expert 4 Nikonova, who became famous for her conclusion on Shaninka: of her 22 articles, only three were published in the RSCI core; half of its links are from co-authors; There are publications in dubious journals, including the Mediterranean Journal of Social Sciences. Among experts there are more people who publish a lot on average but few in top journals, while among non-experts there are more people with fewer publications but in higher quality journals. It is likely that this pattern of publications—increasing quantity at the expense of quality—is characteristic of managers of Russian universities, who, while pursuing an administrative career, nevertheless consider it necessary to create the appearance of scientific work (75% of experts occupy a leadership position at the university).
Experts are presented as the best representatives of the university environment, although in terms of their characteristics they are much worse than the average Russian teacher. Among the experts, we find a significant proportion of professional imitators of academic activity, who are extremely unscrupulous with regard to their own scientific achievements. It is these people who are more energetic than others in the work of creating the appearance of compliance of the university with the wishes of the ministry. The agency has to choose among those who have volunteered to become Rosobrnadzor experts - to do this, you need to write an application yourself and then pass an exam on your knowledge of legislation. Usually there is no strict selection - according to the service, in 2016, 11 qualifying exams were held, according to the results of which 138 people were certified as experts; only 20 applicants were denied certification. The absence of strict selection largely leads to the results that we saw. If he had been, then Marina Vladimirovna Ulyanova, author of the periodic system of General Laws of Management and bioadequate teaching methods, co-chairman of the regional department of “Noosphere Education” of the Russian Academy of Natural Sciences, rector of the Institute of Holodynamics (“holodynamics” - from English) would not have been listed among the experts in the past whole , i.e. “whole”).
If Rosobrnadzor refuses almost no one, this means that there is negative self-selection - documentation specialists who represent not the strongest universities, who are great professionals in how to imitate science, are asked to become experts. They may have their own interests, but more importantly, there are serious institutional reasons for this. Our interviews show that universities are setting up their own teams of experts, who should become internal Rosobrnadzor, training the university to pass real checks. First of all, universities see this as a way out, as they themselves fall under the radar of Rosobrnadzor. Small private universities are “overrepresented” in the list of experts - they employ 132 experts (19%), while only 6.7% of all teachers, according to 2014 data, worked in private universities. Of all the private universities, the most noticeable is the Eurasian Open Institute, which employs six experts from Rosobrnadzor (and the number of full-time teachers at the Eurasian Open Institute is only 39 people).
The activities of Rosobrnadzor contributed to the formation of a stable professional group ready to cooperate in control and supervision. Teachers who do not have a strong desire to do meaningless paperwork instead of teaching and research are not eager to be included in the list of experts. It follows from this that it is necessary not only to change the method of selecting experts, but to change the entire control system, if not completely abolish it.
Katerina Guba, Alexandra Makeeva,
Mikhail Sokolov, Angelika Tsivinskaya
2 rosvuz.dissernet.org/vuz/131136