It’s been a rough few weeks for hundreds of companies across the globe who operate their businesses with VMware’s ESXi server.
A two-year-old Remote Code Execution (RCE) vulnerability is being used against unpatched servers to allow hackers to deploy the new ransomware variant called ESXiArgs.
A hackers two favourite words
The vulnerability is tracked as CVE-2021-21974, and affects OpenSLP as used in ESXi. The vulnerability is described as a heap-overflow vulnerability, whereby a malicious actor residing within the same network segment as ESXi who has access to port 427 may be able to trigger the vulnerability resulting in the ability to run arbitrary code.
VMWare issued a patch in February 2021 upon discovery of the vulnerability, however, the vendor is urging customers to immediately apply the patch if the ESXi hypervisor has not yet been updated.
What is ESXi server?
ESXi is a component of the VMware vSphere suite, which is a cloud computing platform for managing virtual infrastructure and is a type of hypervisor software that provides virtualization services to create and manage virtual machines (VMs).
What is OpenSLP?
OpenSLP (Open Service Location Protocol) is (as the name suggests) an open-source implementation of the Service Location Protocol (SLP), which is used for discovering network services such as printers, file servers, and web servers in a network environment.
OpenSLP is used to build distributed network environments and simplify network administration. The automatic discovery of network resources and services, allows administrators to manage them more efficiently.
OpenSLP also supports the integration of new services and devices into the network without manual configuration, making it easier for administrators to manage the growth of their network.
What is the vulnerability?
This vulnerability targed in this attack, allows network-adjacent attackers to execute arbitrary code on affected installations of VMware ESXi simply by sending specifically crafted data to port 427 of an unpatched server.
The flaw in the deployed variant of OpenSLP exists within the processing of recieved SLP messages.
Improper validation of the length of the user-supplied SLP data prior to copying it to a heap-based buffer can allow an attacker to execute code in the context of the SLP daemon.
Authentication to the server is not required to exploit this vulnerability.
Affected systems
ESXi versions which are known to be impacted by this vulnerability include:
- 7.0 (before ESXi70U1c-17325551)
- 6.7 (before ESXi670-202102401-SG)
- 6.5 (before ESXi650-202102101-SG)
What does the attack do?
As previously mentioned, the vulnerability targeted in the attack allows for remote code execution (RCE) which could allow for almost any type of attack. In this case, it is currently being used to deploy ransomware.
The ransomware in this campaign targets virtual machine volumes within the /vmfs/volumes directory, encrypting their virtual disks and memory files. The encryptor is executed by a bash script placed in /tmp/ as part of the attack.
The execution flow of the script is as follows:
- Set the staging directory to /tmp/ and upload the following attack files:
- encrypt – The ransomware binary
- encrypt.sh – The deployment script, which contains the commands executed as part of the post-exploitation phase
- public.pem – A secondary public RSA key used to encrypt the primary encryption key
- motd – The ransom note displayed as the “message of the day”
- index.html – The ransom note (HTML), which replaces the home page of VMware ESXi
- Set hard limits on the pipe buffer size (
ulimit -Hp) and maximum open file descriptors (ulimit -Hn) - Modify the configuration files for the present virtual machines by corrupting the references to their corresponding .vmdk and .vswp files
- Kills all VMX processes
- Encrypt all virtual machine files within the /vmfs/volumes directory, targeting files with the following extensions: .vmdk, .vmx, .vmxf, .vmsd, .vmsn, .vswp, .vmss, .nvram, .vmem
- Replace the VMware ESXi home page with the HTML ransom note
- Replace the server’s “message of the day” (motd) banner with a ransom note
- Deletes all discovered .log files from the server
- Overwrites /sbin/hostd-probe with /sbin/hostd-probe.bak
- If the ESXi build number is detected as 7 or above, lines 1-8 and the last line are deleted from the /var/spool/cron/crontabs/root file. The file is also timestomped to conceal its modification. This deletes the default cron jobs on the host.
- For build numbers other than 7, the first line from /bin/hostd-probe.sh is deleted. This file is subsequently timestomped to conceal its modification.
- Deletes the file /store/packages/vmtools.py.
- Deletes the last line of the /etc/vmware/rhttpproxy/endpoints.conf file and then timestomps the file to conceal its modification.
- Deleting this line removes access to the ESXi management API.
- Overwrites the /etc/rc.local.d/local.sh file with blank content and then timestomps the file to conceal its modification
- Deletes the following attacker files from /tmp/: encrypt, nohup.out, index.html, motd, public.pem, archieve.zip
- Executes the /bin/auto-backup.sh script.
- Responsible for backing up the ESXi configuration, which is then restored upon reboot
- Executes /bin/rm — “$0” to delete the running attacker script
- Executes /etc/init.d/SSH start to start the SSH server on the host
There is was hope!
When encrypting a device, the encrypt.sh script looks for virtual machine files to encrypt. For each file that is found, the script checks the file size, and if the file is smaller than 128 MB, encrypts the whole file in 1MB increments.
However, for files larger than 128 MB, it computes a ‘size_step,’ which causes the encryptor to alternate between encrypting 1 MB of data and not encrypting chunks (the size_step in megabytes) of data.
This means for a 4.5 GB file, it would generate a size_step of ’45,’ causing the encryptor to alternate between encrypting 1 MB of the file and skipping 45 MB of the file. So, as you can see, quite a bit of data remains unencrypted by the time it’s finished encrypting a file.
For even larger files, like a 450GB file, the amount of skipped data rises dramatically, with the size_step becoming ‘4607,’ now alternating between encrypting 1MB and skipping 4.49 GB of data.
Due to these large chunks of unencrypted data, researchers devised a method to recover virtual machines using the large and primarily unencrypted flat files, where the virtual machine’s disk data is stored.
The US Cybersecurity and Infrastructure Security Agency (CISA) has released a recovery script for victims of the ESXiArgs ransomware to automate the recovery process
CISA’s ESXiArgs-Recover tool is available for free on GitHub and organizations can use it to attempt the recovery of configuration files on vulnerable VMware ESXi servers that the ransomware variant might have encrypted.
However….
A second ESXiArgs ransomware wave includes a modified encryption routine that encrypts far more data in large files.
The updated ESXiArgs malware has not altered the encrypter, but the encrypt.sh script’s ‘size_step’ routine had been taken out and simply set to a value of 1.
This update causes the encryptor to now alternate between encrypting 1 MB of data and skipping 1 MB of data.
All files over 128 MB will now have 50% of their data encrypted, making them likely unrecoverable.
This change also prevents the previous recovery tools from successfully recovering machines, as the flat files will have too much data encrypted to be usable.
This second wave of attack also made a minor change to the ransom note by no longer including bitcoin addresses in the ransom note, as shown below.
The removal of the bitcoin addresses was likely due to them being collected by security researchers to track ransom payments.
