{"id":7244,"date":"2023-04-01T19:46:56","date_gmt":"2023-04-01T18:46:56","guid":{"rendered":"https:\/\/www.kevinmaschke.com\/?p=7244"},"modified":"2025-12-02T00:57:12","modified_gmt":"2025-12-02T00:57:12","slug":"esxiargs-ransomware-2023-recovery","status":"publish","type":"post","link":"https:\/\/www.kevinmaschke.com\/en\/esxiargs-ransomware-2023-recovery\/","title":{"rendered":"ESXiArgs Ransomware (2023) &#8211; How to recover a VM with no snapshots"},"content":{"rendered":"\n<p>This past February, <strong>thousands of ESXi servers were affected by a new ransomware<\/strong> attack (called ESXiARGS) that targeted the servers through a security flaw (tracked as <a href=\"https:\/\/www.vmware.com\/security\/advisories\/VMSA-2021-0002.html\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">CVE-2021-21972, CVE-2021-21973 and CVE-2021-21974<\/a>), which is caused by a heap overflow issue in the OpenSLP service. The systems targeted appeared to be ESXi hypervisors in version 6.x and prior to 6.7.<\/p>\n\n\n\n<p>In this article we will go over the steps to recover VMs that <strong>DO NOT have<\/strong> snapshots (<code>-delta<\/code> or <code>-sesparse<\/code> files), only a <code>flat<\/code> VMDK file. At this point, it does not seem like there is a process that works for everyone to successfully recover full VMs or their data if these had snapshots (<code>-delta<\/code>) files. Some people seem to have been able to recover VMs that had <code>-sesparse<\/code> snapshots.<\/p>\n\n\n\n<p>I&#8217;m going to assume that you already have stopped the attack and regained control on your ESXi Hypervisor, and only concentrate on recovering the virtual machines.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Recovery Prerequisites<\/h2>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background\" style=\"font-style:normal;font-weight:300\"><strong>Disclaimer<\/strong><br>These steps have worked for some people, but not for everyone. The probability of success depends on how much of your VM disk has been encrypted, which depends on how long the encryption script ran and\/or if it failed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Backup<\/h3>\n\n\n\n<p>The first thing to do, before anything else, create backup of the VM files:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Either create a copy of the actual VM folder, or<\/li>\n\n\n\n<li>create a backup folder inside the VM folder and copy all files there.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Remove unnecessary files<\/h3>\n\n\n\n<p>The following files won&#8217;t be necessary for this process, and can safely be deleted. You should still have a copy of all of them in the backup you created previously, just in case.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>.args<\/li>\n\n\n\n<li>.vmxf<\/li>\n\n\n\n<li>.vmsd<\/li>\n\n\n\n<li>.vmsn<\/li>\n\n\n\n<li>.vmem<\/li>\n\n\n\n<li>.nvram<\/li>\n\n\n\n<li>.vmx &#8211; you restore it from the .vmx~ file.<\/li>\n<\/ul>\n\n\n\n<p>You can remove them and recover the <code>vmx<\/code> file using the following commands:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">[root@esxihost:]$ mv VMNAME.vmx~ VMNAME.vmx\n[root@esxihost:]$ rm *.args *.vmfx *.vmsd *.vmsn *.vmem *.nvram<\/pre>\n\n\n\n<p>Now your folder should look a little bit cleaner.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">ESXiArgs &#8211; VM Recovery<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Recreate VMDK file<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Note down the original size of the <code>-flat<\/code> file using the command <code>ls -la<\/code>:<\/li>\n<\/ol>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">[root@esxihost:]$ ls -la\ntotal 92925984\ndrwxr-xr-x    1 root     root          1120 Feb 26 18:01 .\ndrwxr-xr-t    1 root     root          2100 Feb  4 15:56 ..\ndrwxr-xr-x    1 root     root          3500 Feb 20 16:31 BKUP\n-rw-------    1 root     root  107374182912 Feb  3 10:18 VMNAME-flat.vmdk\n-rw-------    1 root     root          1045 Feb  3 10:18 VMNAME.vmdk\n-rwx------    1 root     root          3449 Dec  5 17:12 VMNAME.vmx<\/pre>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li>Delete the existing <code>.vmdk<\/code> file by typing <code>rm -rf xxx.vmdk<\/code>:<\/li>\n<\/ol>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">[root@esxihost:]$ rm VMNAME.vmdk<\/pre>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li>Create a new \u201ctemp\u201d disk (<code>-flat.vmdk<\/code> and <code>.vmdk<\/code>) using the <code>vmkfstools<\/code> command. Make sure the size you specify is exactly the same as the size of the original <code>-flat<\/code> file when running <code>ls -la<\/code>:<\/li>\n<\/ol>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">[root@esxihost:]$ vmkfstools -c 107374182912 -d thin temp.vmdk\nCreate: 100% done.<\/pre>\n\n\n\n<ol start=\"4\" class=\"wp-block-list\">\n<li>There should now be two new files, a <code>temp-flat.vmdk<\/code> file and a <code>temp.vmdk<\/code> file. The original and new <code>-flat.vmdk<\/code> files should have the exact same size.<\/li>\n\n\n\n<li>Edit the <code>temp.vmdk<\/code> file:\n<ul class=\"wp-block-list\">\n<li>On line 9, replace <code>temp-flat.vmdk<\/code> with the name of the original <code>-flat<\/code> file, and<\/li>\n\n\n\n<li><strong>If your disk WAS NOT thin provisioned<\/strong>, remove or comment out line 19 <code>ddb.thinProvisioned<\/code>. If it was, leave the line as is.<\/li>\n\n\n\n<li>Save the file.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Disk DescriptorFile                                                                                      \nversion=1                                                                              \nencoding=\"UTF-8\"                                                                       \nCID=fffffffe                                                                           \nparentCID=ffffffff                                                                     \ncreateType=\"vmfs\"                                                                      \n                                                                                       \n# Extent description                                                                   \nRW 209715201 VMFS \"WebServer_2-flat.vmdk\"                                              \n                                                                                       \n# The Disk Data Base                                                                   \n#DDB                                                                                   \n                                                                                       \nddb.adapterType = \"lsilogic\"                                                           \nddb.geometry.cylinders = \"13054\"                                                       \nddb.geometry.heads = \"255\"                                                             \nddb.geometry.sectors = \"63\"                                                            \nddb.longContentID = \"cf65c0c190124ab5571025d1fffffffe\"                                 \n#ddb.thinProvisioned = \"1\"                                                             \nddb.uuid = \"60 00 C2 94 74 7b 9f 6f-36 7e b3 ce d1 90 72 2f\"                           \nddb.virtualHWVersion = \"14\"<\/pre>\n\n\n\n<p>At this point you should have something similar to this:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">[root@esxihost:]$ ls -la\ntotal 92925984\ndrwxr-xr-x    1 root     root          1120 Feb 26 18:01 .\ndrwxr-xr-t    1 root     root          2100 Feb  4 15:56 ..\ndrwxr-xr-x    1 root     root          3500 Feb 20 16:31 BKUP\n-rw-------    1 root     root  107374182912 Feb  3 10:18 VMNAME-flat.vmdk  (ORIGINAL)\n-rw-------    1 root     root          1045 Feb  3 10:18 VMNAME.vmdk       (Recreated &amp; Edited)\n-rwx------    1 root     root          3449 Dec  5 17:12 VMNAME.vmx        (Recovered from .vmx~)<\/pre>\n\n\n\n<p>Let&#8217;s now test these <code>.vmdk<\/code> and <code>-flat.vmdk<\/code> files with a VM. To do this, you&#8217;ll have to do the following:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Create a new VM<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Create a new VM folder (outside of the original VM folder) called \u201cTest\u201d.\n<ul class=\"wp-block-list\">\n<li>You can do this either via the shell (SSH) or the Datastore Browser.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Copy the following files into this folder:\n<ul class=\"wp-block-list\">\n<li><code>VMNAME.vmdk<\/code><\/li>\n\n\n\n<li><code>VMNAME-flat.vmdk<\/code><\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Create a new VM through the UI:\n<ul class=\"wp-block-list\">\n<li>Same or similar CPU and RAM (I don\u2019t think it makes a big difference).<\/li>\n\n\n\n<li>Remove the default Hard Disk, and add a new hard disk, selecting <code>Existing hard disk<\/code>. Then browse to the \u201cTest\u201d folder created earlier, and select the <code>.vmdk<\/code> file in there.<\/li>\n\n\n\n<li>Under \u201cCD\/DVD Drive 1\u201d, select <strong>Datastore ISO File<\/strong> and browse to a linux live CD. In my case I used <a href=\"https:\/\/www.kali.org\/get-kali\/\" target=\"_blank\" rel=\"noopener\"><strong>Kali Linux Live CD<\/strong><\/a>, which worked. Make sure \u201cConnect at power on\u201d is selected.<\/li>\n\n\n\n<li>Start the VM.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20.jpg\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"732\" src=\"https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-1024x732.jpg\" alt=\"ESXi - Create new VM\" class=\"wp-image-7275\" srcset=\"https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-1024x732.jpg 1024w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-400x286.jpg 400w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-100x71.jpg 100w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-1536x1098.jpg 1536w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-500x357.jpg 500w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-374x267.jpg 374w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-851x608.jpg 851w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20-1481x1059.jpg 1481w, https:\/\/cdn.kevinmaschke.com\/wp-content\/uploads\/2023\/03\/24163714\/Screenshot-2023-03-24-at-16.36.20.jpg 1970w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption class=\"wp-element-caption\">Create a new VM but replace the default disk with the disk of the encrypted VM.<\/figcaption><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Recover Partition with Testdisk<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Boot the Live CD &#8211; in my case with graphical interface &#8211; to try to recover your partition and re-install Grub. These steps may vary depending on what partitions you had on your VM.<\/li>\n\n\n\n<li>Open a terminal and use <code>testdisk<\/code> to recover your partition table. I followed <a href=\"https:\/\/linuxconfig.org\/how-to-recover-partition-table-in-linux\" target=\"_blank\" rel=\"noopener\"><strong>this guide<\/strong><\/a> to recover my <code>Linux<\/code> partition. <strong>DO NOT RESTART<\/strong> as indicated in this guide.\n<ul class=\"wp-block-list\">\n<li>Install <code>testdisk<\/code> if it is not installed (it comes with Kali Linux).<\/li>\n\n\n\n<li>Run <code>testdisk<\/code>.<\/li>\n\n\n\n<li>Select the <code>No Log<\/code> option and proceed.<\/li>\n\n\n\n<li>Highlight the VM disk (usually <code>\/dev\/sda<\/code>), and select <code>Proceed<\/code>.<\/li>\n\n\n\n<li>Select the partition type (<code>Intel\/PC<\/code>).<\/li>\n\n\n\n<li>Select <code>Analyse<\/code> and press enter.<\/li>\n\n\n\n<li>Select <code>Quick Search<\/code> and press enter.<\/li>\n\n\n\n<li>Highlight your partition if found (in my case <code>Linux<\/code> &#8211; it might be called different depending on your OS\/setup) and press enter.<\/li>\n\n\n\n<li>If the partition table looks good, choose the <code>Write<\/code> option, and confirm it in the next step by pressing <code>Y<\/code>.<\/li>\n\n\n\n<li>When done, use <code>Quit<\/code> to exit testdisk.<\/li>\n\n\n\n<li>Use <code>fdisk -l<\/code> to list and check your disk and partition.<\/li>\n\n\n\n<li><strong>DO NOT RESTART.<\/strong><\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Re-Install GRUB<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Re-install Grub with the following commands:<\/li>\n<\/ol>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"bash\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">root@kali-live:~# mount \/dev\/sda1 \/mnt\nroot@kali-live:~# mount --bind \/dev \/mnt\/dev\nroot@kali-live:~# mount --bind \/dev\/pts \/mnt\/dev\/pts\nroot@kali-live:~# mount --bind \/proc \/mnt\/proc\nroot@kali-live:~# mount --bind \/sys \/mnt\/sys\nroot@kali-live:~# chroot \/mnt \/bin\/bash\nroot@kali-live:~# grub-install \/dev\/sda<\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background\" style=\"font-style:normal;font-weight:300\">Some guides I found online state to only use <code>chroot \/mnt<\/code> when using the <code>chroot<\/code> command. This did not work for me and I was getting the following errors: <code>chroot: failed to run command '\/bin\/bash'<\/code> or <code>chroot: failed to run command '\/bin\/zsh'<\/code> &#8211; This is because if you don\u2019t specify the path for bash, Kali Linux will attempt to <code>chroot<\/code> using the local shell environment, which cannot work. You need to specify the path and shell used by the OS you\u2019re VM was using.<\/p>\n\n\n\n<p class=\"has-pale-cyan-blue-background-color has-background\" style=\"font-style:normal;font-weight:300\">Some guides I found online state to run <code>update-grub after the grub-install<\/code> command. <strong>I did not do this<\/strong>.<\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li>Exit the <code>chroot<\/code> and reboot the machine.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Boot the VM<\/h2>\n\n\n\n<p>It should boot up the VM\u2019s operating system from the <code>-flat.vmdk<\/code> file. You should be able to login and browse the file structure.<\/p>\n\n\n\n<p>If this worked, the base VMDK and Flat file should be good! \u2705 ?<\/p>\n\n\n\n<p>Hopefully someone manages to find a way to recover Virtual Machines that have snapshots. If that happens, I&#8217;ll write about it!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Additional Information \/ Sources<\/h2>\n\n\n\n<p>The following are useful links for additional information:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.vmware.com\/security\/advisories\/VMSA-2021-0002.html\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">VMWare Advisory<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/blogs.vmware.com\/security\/2023\/02\/83330.html\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">VMWare Blog<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.bleepingcomputer.com\/news\/security\/massive-esxiargs-ransomware-attack-targets-vmware-esxi-servers-worldwide\/\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">BleepingComputer News<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.bleepingcomputer.com\/forums\/t\/782193\/esxi-ransomware-help-and-support-topic-esxiargs-args-extension\/\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">BleepingComputer Forum<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/enes.dev\/\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">Enes.dev<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/blog.ovhcloud.com\/ransomware-targeting-vmware-esxi\/\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">OVH Blog<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/kb.vmware.com\/s\/article\/76372\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">Disable SLP Service<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/kb.vmware.com\/s\/article\/1002511\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">Recreate VMDK Descriptor<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/kb.vmware.com\/s\/article\/1026353\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">Recreate VMDK Descriptor for Delta file disks<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/cisagov\/ESXiArgs-Recover\/blob\/main\/recover.sh\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">CISA ESXiArgs Recover Script<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.cvedetails.com\/cve\/CVE-2020-3992\/\" data-schema-attribute=\"about mentions\" target=\"_blank\" rel=\"noopener\">CVE Details<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This past February, thousands of ESXi servers were affected by a new ransomware attack (called ESXiARGS) that targeted the servers through a security flaw (tracked as CVE-2021-21972, CVE-2021-21973 and CVE-2021-21974), which is caused by a &#8230;<\/p>\n","protected":false},"author":1,"featured_media":7269,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[207,410,219],"tags":[2733,1695,2734],"class_list":["post-7244","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux-en","category-guides","category-technology","tag-esxi","tag-linux-en","tag-ransomware"],"acf":{"book_cover":null,"special_featured_image":null},"_links":{"self":[{"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/posts\/7244","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/comments?post=7244"}],"version-history":[{"count":0,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/posts\/7244\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/media\/7269"}],"wp:attachment":[{"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/media?parent=7244"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/categories?post=7244"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kevinmaschke.com\/en\/wp-json\/wp\/v2\/tags?post=7244"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}