For those who don’t keep up to date with the world of IT – back in late March, Andres Freund, a developer working at Microsoft noticed some odd behaviour in the login process of an SSH connection they used to connect securely to a Linux server so began investigating the reasons why this was happening. The issues were quite small – a 500ms latency difference – and would probably gone unnoticed by most people, but were enough to raise the suspicions of Freund.

On the 29th March, Freund posted a report to the security forum openwall stating that they had found that a Linux package called liblzma which is part of a library of tools called XZ UTILS had been compromised with a serious malicious capability. It was this backdoored package that was causing the issues Freund was experiencing, and this led to the IT and Cyber security world going into full-on panic recovery mode.

The Openwall post from Andres Freund

Initially, the backdoor was discovered on a Linux Debian system, but due to the nature of how Linux distributions tend to include features to support cross-platform compatibility, it transpired that the malicious code would be able to impact every installation of Linux all over the world – regardless of distribution.

What is XZ UTILS?

XZ Utils (previously known as LZMA Utils) is a set of free command-line data compressors for both Linux and Windows operating systems and can compress and decompress the xz and lzma file archive formats

XZ Utils consists of two major components:

  • XZ – the command-line tool
  • liblzma, a software library with an API for other programs to use

Between 2005 to 2008, Lasse Collins and others developed the .xz file format which uses LZMA compression. The tool has now become a core part of many Linux kernels.

The malware

Forensic analysis of the malicious code revealed that a successful exploit could led to undetected remote access for a threat actor with the ability to run arbitrary code on the impacted machine.

This caused the malware to receive a CVSS (Common Vulnerability Scoring System) rating of 10 – the highest value able to be given – this is a BIG deal!

The malware has now been given the CVE entry of CVE-2024-3094.

The malware itself is very clever and contains numerous obfuscation mechanisms to evade detection or analysis – but in simplistic terms, the malware would allow anyone in possession of a predetermined encryption key the ability to include any code of their choice in an SSH login certificate, upload it to a server, and execute it.

In an attempt to graphically describe the complexity of the malware, Thomas Roccia, a threat researcher at Microsoft produced an infographic of how the whole story unfolded:

In itself, this is a very elegant malware, but what has caused the biggest amount of intrigue is the length the threat actors went to in order to get this malware included in the Linux system.

A timeline of Social Engineering

It appears that the story starts back in November 2021 when someone with the username JiaT75 made a code commit to the open source project working on libarchive which was approved and pushed into the code.

This code change replaced the safe_fprint function with a variant that has long been recognised as less secure, but no one noticed at the time.

In the following year, the threat actor, going by the name Jia Tan published a number of patches to the xl-devel mailing list – the group responsible for the maintainance of the XZ Utils package.

One of the messages to the XZ Utils team from Jia Tan

Shortly after the update requests were published, another couple of users – Jigar Kumar and Dennis Ens started pressuring Lasse to allow Jia’s code to be applied to XZ Utils, and also to allow Jia to become a repository maintainer:

Following this pressure, Lasse said that Jia needed to take a greater role in the project going forward, and allowed Jia to become a maintainer of the repository.

Other than in the xz-devel forum, no trace of Dennis Ens or Jigar Kumar can be found online – indicating that these were fake accounts created for the sole purpose of promoting Jia Tans code & involvement in the maintenance of XZ-Utils

In September 2022, Jia outlined the 5.4.0 release of XZ Utils, and, by November 2022, Lasse stated in the README file that the project maintainers were now Lasse Collin and Jia Tan.

January 2024 saw the release of Version 5.4.1 of XZ Utils, and in February 2024 the back door code was hidden in test files, and later deployed as Version 5.6.0 as the xz-5.6.0.tar.gz distribution.

A new backdoor was inserted into Version 5.6.1 and pushed out in March – this is the version which caused issues that raised Freund’s suspicions.

In February and March of 2024, Jia committed two versions of XZ Utils (5.6.0 and 5.6.1) which contained the backdoor code and proceeded to canvas the developers of Ubuntu, Red Hat, and Debian to integrate the new version with their operating systems.

The affected Linux distros are:

At the moment, no-one knows who is behind this complex, yet elegant attack on Linux, and it is doubtful we ever will as the finger of suspicion is pointing to one of a number of government agencies around the world as they are the only ones with the resources, time, funding, and most of all desire to be able to compromise the worlds most widely deployed operating system.