ESXi on old hardware Error 10 – Out of resources failed to malloc MMIO

At home I use a old Dell Precision T5600. It has 16 cores total with 128GB of memory and while old its still good for running multiple workloads via ESXi. In the 6.x days I had no issues with ESXi, however with 7.0 I now get the CPU is unsupported as well as many other warnings. However I was able to run 7.0 Update 2 build 17867351 without issues. When I first installed ESXi I used BIOS to install and to run it since it never worked with EFI and at the time I did not bother to figure out why.

A week ago my fingers got itchy and wanted to upgrade ESXi to the latest version (currently 7.0U3g-20328353) so I performed the upgrade using the steps outlined on https://www.gooksu.com/2022/01/update-esxi-7-x-using-esxcli/

Everything went fine but the host never booted up so I went to check it out and saw this

Requested malloc size 25778144 failed, caller offset Ox6d591d77
Out of resources For decompressing data(25778124)
gzip extract failed for usanngnt.v00 (size 4812529): Out of resources
Error 1D (Out of resources) while loading module: usanngnt.v00
Compressed MDS: 3efb9779f9b4f9f4f9a87aleb779bc19
Decompressed MD5: OODDDDOOOODDODOOOOOODOODODODODOO
Fatal error: 10 (Out of resources)

Bummer!

I remember seeing some posts about adjusting the MMIO settings and tried all different combinations but could not fix it.

I did however see that some of the older hardware might have issues with EFI if the partitions are FAT16, so I decided to dig a bit

I created a linux usb stick to get into a linux rescue mode. You can do this with a live linux or other linux based live usb stick. I happened to have a CentOS ISO so I created a boot stick using the CentOS ISO and booted into the rescue mode.

Took a look at the partition table

I see that my boot disk is sda and when I looked at the partition tables it shows that it is fat16

so I will perform steps to backup sda1 and reformat it as FAT32

# cd /mnt
# mkdir sda1 tmp
# mount /dev/sda1 /mnt/sda1
# rsync -ravt /mnt/sda1/ /mnt/tmp/
# umount /mnt/sda1
# mkfs.vfat -F32 -s 2 /dev/sd1
# mount /dev/sda1 /mnt/sda1
# rsync -ravt /mnt/tmp/ /mnt/sda1/

I added the -s 2 options for mkfs.vfat since when I ran without it I got an error for Chunks

Now when I do the listing I get

I changed everything in the BIOS to boot from EFI and rebooted ana TADA~ everything is working fine!

When you upgrade the ESXi host and it has issues booting up - all you have to do is follow this process again to have it boot. Normally it will just sit on a blank screen after the machine POSTs