A burnt programmer and toxic colleagues. How one of the largest hacker attacks in the world did not happen

Attackers almost got the opportunity to control most servers in the world. To do this, they needed to find a burnt creator of a small data to compress data

XZ

- And to hide it with toxic comments about poor -quality and slow work.

On March 28, programmer Andres Froind, one of the most popular Postgres database management systems and Microsoft employee, noticed that his computer works a little slower than usual. To be precise, the connection to the server took half a second longer. On the computer Froinda stood the latest version of Debian.

Andres decided to figure it out and found that an unusually many resources of his computer are spent on SSHD - this is one of the most common programs for connecting to the server via the Internet.

He recalled that he had recently read about other oddities in the latest versions of Debian . Andres began to study changes in these versions - and discovered a hacker attack, which was preparing for more than two years and could hack millions of servers around the world.

What is XZ and why almost everyone uses it

Soon, Andres Froind discovered that in fresh versions of the XZ package - this program is included in the standard DEBIAN set - Backdor appeared. Backdor is a vulnerability laid down by the developer, which allows you to circumvent protection mechanisms and gain access to the computer. The version with Bacdor began to spread in February 2024.

XZ is a set of data to compress or unpack data using the LZMA algorithm. In certain situations, this algorithm works much more effective than other compression methods. Windows users can be familiar with the 7-ZIP program. The author of this program Igor Pavlov came up with the LZMA algorithm. In fact, XZ is a version of 7-ZIP for operating systems based on Linux .

The XZ was developed by the programmer and Cipheropank Lass Collin . In 2005, he wanted to make his version of Linux, which could fit on a CD with a capacity of 700 megabytes.

The assembly of Linux from Collin did not become popular, but in the process of working on it he created a set of convenient tools for compressing and unpacking data - just based on the algorithm of Igor Pavlov. This set - XZ Utils - turned out to be so successful that it entered the standard equipment of most popular Linux distributions, including Debian , Fedora , Ubuntu and Gentoo . Code XZ is even included in the core of the Linux operating system.

In addition, XZ began to be used in the work of other key programs, for example, in the program already mentioned above for connecting to SSHD servers.

Preparation for the attack: how to hide the developer

Code XZ was published on the GITHUB service, this is the largest community of developers of open source programs. Anyone could send Lasa changes or improvements for the XZ , which he had to approve and publish, after which the new version fell into the operating systems.

In history, hacker attacks have already occurred, during which attackers gained access to the repository on GitHub , including through social engineering.

In the story with Backdor in XZ, hackers used just the psychological state of the creator of the program, to the loosening of which they themselves clearly had a hand.

On October 29, 2021, the developer with the nickname Jia Tan sent Lass Collin his first editing for XZ , exactly a month later- the second , and in April 2022 there were several more. Then, letters from the user with the nickname Jigar Kumar began to appear in the postal newsletter of XZ: he complained that Collin accepted corrections from the Jia Tan developer for too long.

“You, of course, are well done, but with such a slow schedule of updates, unfortunately, years will pass before the community receives this correction that improves life,” wrote Jigar Kumar .

Another user joined him, Dennis Ens , who was dissatisfied with the fact that no one supported the XZ version for Java . Lass Collin in the answers apologized for delays and complained of his psychological state, fatigue and burnout, but Jigar and Dennis continued to poison him for slowness and began to hint that it would be nice to transfer the project management to someone more active.

“With your current speed, I doubt that version 5.4.0 will be released this year. The only progress from April is several small changes in tests. You ignore the fact that a bunch of corrections rot in this newsletter. You strangle your project. Why should you wait for 5.4.0 to change the project manager? Why postpone what the project needs? " - wrote Jigar Kumar in June 2022.

After the opening of the attack prepared through the XZ , it turned out that Jigar Kumar and Dennis Ens probably never existed: they left no traces on the Internet at all, their emails on the Mailhost domain were no longer used anywhere, they did not participate in any other postal newsletters, except XZ . The only thing these users did was complained about Lass Collin's slowness and put pressure on it.

At the end of June 2022, the developer surrendered and agreed that Jia Tan is more suitable for the role of the project manager. At this point, the Creator XZ had already accepted many patches from Jia Tan and wrote in the newsletter that this user would “get a big role in the project in the future”. At the end of November 2022, Lass Collin partially handed over to him the right of access andindicated his mailing address in the form for feedback.

January 11, 2023 Collin published his latest version of XZ under number 5.4.1; Version 5.4.2 in March 2023 came out under the authorship of Jia Tan .

Having gained control over XZ, the crackers add vulnerability there

Jia Tan really continued to work on XZ : he made his own and other people's changes and even thanked other developers for their contributions. Lass Collin himself also continued to work on the package sometimes, but began to leave for regular holidays without the Internet. The main decisions were now made by the new owner. Jia Tan eventransferred the project from the Lass home website to Github .

Modern developers are trying to automate testing their programs in order not to depend on human mistakes and not waste time on routine. Lass Collin was an old school developer and tested everything himself. Jia Tan wrote automatic tests for XZ . It was under the guise of one of the tests on February 23, 2024 that headded there the files containing Backdor. The next day, Jia Tanpublished a vulnerable version 5.6.0, and she began to diverge through the distributions of Linux .

The first version of Backdor was faced with the problem: automatic tests in two Linux - Gentoo and Fedora distributions were ceased to be performed . This attracted attention to changes and caused discussions in the community of programmers. It was these discussions that Andres Froind recalled when he noticed a slowdown in the speed of his computer.

The unstable version of the XZ began to be excluded from other popular libraries, and Jia Tan had to hurry. On March 9, hereleased the corrected version 5.6.1.

The still vulnerable, but less suspicious update 5.6.1 also began to penetrate into test versions of Linux distributions. Sometimes Jia Tan himself asked to include it in updates, sometimes another user did it, Hans Jansen . As in the case of the Commentators of Lass Collin, this user has not yet been found anywhere; His only activity, in addition to such comments, is the corrections to the XZ , at first glance harmless, but laid the foundation on which the backer from Jia Tan works.

By the time Microsoft developer Andres Froind discovered this backdor, he was already published in the beta versions of Debian and Fedora . Froind first contacted the developers of distributions, and the next day he published a post from which the whole world learned about a hacker attack.

How this backdor works

The changes that Jia Tan made to the code XZ affect the SSHD program, which allows the servers to send any teams.

For a login, either the user name and the password are used for the server, or - what happens more often - the certificates or keys of the client, which are pre -known to the server, which are almost impossible to choose.

Backdor forced SSHD to accept not only original certificates, but also specially prepared Jia Tan certificates that would give him unlimited access to any system where his version of the XZ package is established.

In fact, the sender of such a certificate became the full owner of any vulnerable server: he could change, copy and delete any data on it, install and delete programs. Unlimited access allows the hacker to hide all their traces, so for a long time the owners would not notice the invasion.

If a special certificate was not received, SSHD worked as usual.

In addition, after installing the infected version, the malicious code could be updated by himself - regardless of whether the user would update the XZ package in the future.

Backdor himself was carefully masked and used several layers to hide the dangerous code.

Possible consequences: full access to millions of servers

Thus, potentially hackers could get unlimited access to millions of servers around the world using SSHD . The intentions of hackers are unknown, but they could become catastrophic for the Internet.

According to W3Tech , more than 80% of sites in the world operate on servers with operating systems based on Linux . All of them could become vulnerable to hackers if Bekdor got into the main branches of updates. By a happy chance, Backdor was found at an early stage when he fell only in the earliest test versions of several large distributions. For example, the most popular Ubuntu distribution did not have time to add it.

“Openssh operates for almost 20 million IP addresses today, it is almost ten times more popular than RDP. If someone was able to create a widespread Backdor for him, it would be bad, ” wrote Kevin Bomon security specialist in his blog.

After discovery, this vulnerability was introduced into the national base of the US vulnerabilities (NVD) with the highest possible level of danger (10 out of 10). The largest Linux distributions issued corrections that returned the version of XZ by the time Jia Tan have not yet received control over the project. Lass Collin had to return from the “Digital Vacation” to restore access to the project and block Jia Tan .

My computer is in danger? Most likely not. Only a few of the latest test versions of the Linux distribution managed to get a vulnerable update.

First of all, the attack was aimed at the server on which the SSHD program was launched. If you work with servers or use the operating system based on Linux , then the easiest way is to make sure that you do not have a compromised version: 5.6.0 or 5.6.1. The list of popular distributions in which the infected version has fallen can be seen here .

Unfortunately, this applies to some MacOS users. The infected version did not fall into the operating system itself, but managed to get into updates to the popular package manager for MacOS - Homebrew . Moreover, XZ is part of many other programs that are often installed through Homebrew , for example, Python programming language. The authors of Homebrew note that, most likely, there are no threats to users, but still do not advise using the compromised version.

For verification, enter the following command in the terminal:

Strings Which XZ | Grep '5.6. [01]'

If this team does not display anything, then, most likely, everything is in order and you do not have an infected version.

If in response you get a line with the version of the program, then you need to immediately roll back the version of at least 5.4.6 in any way suitable for your operating system, and then restart the system (or SSHD ). In the case of Homebrew , just start the command: Brew upgrade .

For a more reliable and serious check, it is better to use the XZ.fail service specially created for this or the XZBOT project.

Who is behind the attack? It is still unclear

It is not known who was hiding behind the pseudonym Jia Tan , and it is unclear even, one is a person or a group of hackers. On GitHub, this user has made changes to dozens of other programs - now they are being studied by security experts.

One of the first actions of this user on GitHub is the changes to the Libarchive library back in 2021. This is a small and harmless correction at first glance: it changes the error message displayed on the user's screen - such trifles are often accepted without proper attention. However, at the same time, Jia Tan replaced the SAFE_FPRINTF function with its unsafe analogue FPRINTF , later it could be used by a hacker. The vulnerability was fixed on March 29 after the discovery of Backdor in XZ .

Researchers studied Jia Tan's operating time - all its changes in XZ are made on GitHub , that is, public . The hacker tried to impersonate the Chinese: he chose a similar to the Chinese name and used the Chinese time zone in most commits.

Despite this, in his code was also found editing in the Eastern European time, and with such intervals for which a person, obviously, could not reach Eastern Europe. Researchers drew attention to the fact that the author worked during the main Chinese holidays, but never on Catholic Christmas on December 25 or on New Year's holidays on December 31 and January 1.

Editors: Maxim Litavrin, Yegor Skovoroda