No tragedies: why Yandex.Zen blocks the content of funds

Yandex.Zen is not ready to show tragic content. The service limited the screening of the materials of the Channel for the Fund for the fight against leukemia, but then admitted that this happened by mistake. Nevertheless, the restriction on “tragic” and “toxic” publications is still valid. Julia Kalenkova, journalist of the greenhouse, decided to understand the wording, as well as analyze international experience.

What happened?

On April 3, 2019, an employee of the Foundation Elena Radchenko announced on her Facebook page about the restriction of publications in Yandex.Zen - the service of personal recommendations from Yandex. This was "associated with the use of materials containing tragic content." In the comments on the post, representatives of Yandex recognized the error, but confirmed: "restrictions can occur in case of toxic content, attempts to speculate on tragedies."

The service does not want users to “specify the attention of readers, publishing posts about tragedies,” since “not everyone wants to see publications that cause negative emotions.” At the same time, the rules indicate that such materials are appropriate if the publication has a “weighty basis for tragic details and a context is necessarily given”.

“It seems to us that the threat of their life is a fairly good reason to publish these stories,” said Elena Radchenko. It is worth emphasizing: the foundation’s channel had the stories of life and illness with portraits of heroes, a description of the work of volunteers, and methodological materials . Similar content is published in other channels, for example, in the diary of the cancer “ Cancer is not a sentence ”.

The number of views on the fund’s channel usually ranged from tens to hundreds of thousands, some publications gained more than a million views (after blocking coverage fell to hundreds of views). Everyone who is familiar with charity is known: to draw attention to each collection or request for help critically, and the score sometimes goes to the clock: “The faster we will collect funds, the more chances that we can save human life. Moreover, we do not publish shocking photographs, terrible descriptions, we have very good journalists who write competently, efficiently and carefully, ”Elena Radchenko commented on the situation in an interview with such matters.

By the way, the articles of the publication “Such cases” (the information portal of the Charitable Foundation “Need Assistance”) were also limited to show.

The requirements for publications by shocking content mean a description or image:

• victims of disasters and violent actions (people and animals);
• injuries, diseases, uglies, corpses;
• the process of childbirth;
• insects and animals whose species causes disgust.

The theme of " for adults " is a separate reason for discussion. The image of art objects (sculpture, painting, graphics) is allowed if the content “does not contain open intimate parts of the body”. You can publish photos of women in swimsuits "provided that the model in the photo does not accept causing poses." Frank visual content includes paintings made in the genre of hyperrealism, as well as photos in the Nude genre.

How is content filtered?

Informing changes in the rules of content monetization (April 2018), Yandex Zen gave examples of publications with impressions (advertising is removed):

• “A man tried to commit suicide in Baku”;
• “Media: five people died in a fire in a correctional center in Peru”;
• “Under Nizhny Novgorod will be judged by obstetrician for the death of a woman in labor with a child”;
• "Under Khabarovsk, 26 pairs of severed hands of non -criminal origin were found."

“We believe that neither the authors nor the platform should earn money on tragic events,” said the media director Yandex.Zen Daniil Trabun. We are talking about materials that describe tragic incidents, disasters, terrorist acts, military operations, military-political conflicts, various types of violence, conspiracy theory and other publications in the language of enmity and hatred.

The greenhouse turned to the Yandex.Zen support service to update the information and deal with disputed formulations. It turned out that tragic and toxic content "falls under different definitions." However, it is proposed to deal with subtleties on their own, studying the requirements for publications . All materials are tested by the system. In order to rehabilitate the channel after blocking or cutting of publications, it is necessary to “fix articles that violate the platform rules, write at least three new articles and inform the support service that the channel is ready to check.”

The most interesting was the question of the work of content filters. It turned out that “there is no list of stop words to Yandex Zen. In the analysis of images, the system and its option of computer vision are involved. ” Obviously, this system should independently “understand” with the subjective assessments “tragic”, “toxic”, “causing” (pose). According to the support service, “everything will depend on the full content of a particular material. Also, an accurate answer can only be given on the example of a particular article, after it passes the filters of the system. ”

Lilia Zemnukhova is a sociologist, a researcher at the Sociological Institute of the Russian Academy of Sciences and the Center for Science and Technology of the European University in St. Petersburg, assessed the situation as follows: “To form a database, you need to make some initial marking, register the rules of recognition (texts and images). Here, at some stages, a person who makes decisions important for the subsequent work of the algorithm is involved. Even if there are no final list of stop words, there are boundaries of what is acceptable for censorship and what is not. ”

According to the expert, it is on these boundaries that the recognition system is oriented - to accept or reject, skip or send to the ban. Given that it is not known exactly how the “ black box ” of Yandex.Zen is arranged, it can be assumed that the frames are determined through the thesaurus and semantics. In other words, someone makes decisions about what exactly will be “sewn up” into the system, and how it determines what exactly to look for semantic units (phrases, sentences, paragraphs or an article entirely).

“If the model is hierarchized (for example, words are first analyzed, then context), then the failure can occur at simple levels. That is, the system may not recognize the entire context of the article by banning it on the basis of a word or sentence. In addition, machine recognition can work even worse with social issues, since issues are discussed truly sensitive and negative from the point of view of the thesis. For example, even a quantitative factor in the mention of “negative” markers can affect, ”explains Lilia Zemnukhova. In her opinion, moderation in such cases is necessary. If the boundaries and markers are not marked very correctly or not optimized taking into account the shortcomings found (as a result, Yandex Zen ultimately apologized), then the accumulated effect causes more rude errors and banks.

Any text arrays of data (unless they are strictly recorded from the very beginning) require constant refinement and identification of subtle differences, because mathematical rules for working with texts miss both context and meanings. Lilia Zemnukhova

International experience

After a mass execution in the mosques of New Zealand (attacking
He broadcast murders on Facebook, and the social network did not manage to stop the distribution of the record) Australia authorities decided not only to limit the turnover of weapons, but also to determine the punishments for social networks that do not remove cruel content. The parliament adopted the law on the fight against content on social networks, which demonstrates cruelty. We are talking about videos depicting terrorist acts, murders, assassination attempts, torture, rape or abduction. If such materials are found, a fine of up to 10.5 million Australian dollars (7.5 million US dollars), or 10 percent of the annual turnover, faces. The leaders of these companies or those who “provide content services” can be punished by three years of prison.

Since the beginning of 2019, the UK government has been discussing the relationship of suicide growth among adolescents with the influence of social networks. The authorities intend to introduce new requirements for social networks in order to protect minors from dangerous content on the Internet. The reason for this was Molly Russell . A 14-year-old girl committed suicide in 2017. After a year and a half, in an interview with the BBC, her father said that Instagram and the Pinterest service are partly guilty of the death of his daughter. The main argument was the content that was interested in the girl on these platforms, namely images on the topic of self-overflow (self-charm) and suicide. According to him, partial wine lies not on the Instagram itself, but on the algorithms of social networks. When the system understands that a person likes a certain content, he begins to more often offer similar through the tape and hashtags.

In almost every country, the rules for filtering content apply. In Germany, for example, Google, Yahoo! And other services are subordinate to the agreement “ Voluntary self-control for multimedia service providers ” (Voluntary Self-Control for Multimedia Service Providers). The list of filters is determined by the Federal Department of Mediasurs, harmful to young people. However, restrictions on exclusively blog platforms and social networks are now only formulated.