Grid.org

As the number of supported platforms (OS and Arch) for my software projects has increased I have been using VM's as development platforms more frequently. Personally I like VMWare's suite of free products as a VM solution and with their recent release of ESXi, I am even more impressed. 

While ESXi has had its issues...such as the recent one of slashdot fame where you couldn't power on your VM's if the hosts date was Aug 12, 2008 or later, it is still a solid platform with a tremendous feature set. The biggest problem we ran into was the hardware support. As we had just bought 4 new Quad Core 4GB RAM boxes to run VM's (orginally we were using vmware server) we wanted to use ESXi. Our boxes had the intel ICH8 chipset, though, and this is unsupported in ESXi, so we had to do a little hacking.

The first step was creating a live USB stick from the ESXi iso...this isn't too tough. You need a recent linux box with "syslinux". I used a Fedora 8 box, so the steps I followed assume that...

1. Plug your USB stick in and mount the root partition (in my case 'mount /dev/sde1 /mnt/usb')
2. Mount the ESXi iso locally (in my case 'mount -o loop esx-iso.iso /mnt/iso')
3. Copy all of the files from the iso to the usb stick (cp -r /mnt/iso/* /mnt/usb)
4. Copy the isolinux.cfg file to syslinux.cfg on the usb stick (cp /mnt/usb/isolinux.cfg /mnt/usb/syslinux.cfg)
5. Unmount the iso and the usb stick (umount /mnt/iso; umount /mnt/usb)
6. Initialize the MBR of the stick (cat /usr/lib/syslinux/mbr.bin > /dev/sde)
7. Run syslinux on the usb stick (syslinux /dev/sde)

At this point your USB stick will boot the stock ESXi image. Now we can easily modify the installation media without burning new cd's! (I even did this on an IPod shuffle so I can listen to music and install ESXi server :) )

The easiest way to get through the install is to update the oem.tgz file on the usb stick...you need to take to files from binmod.tgz (/etc/vmware/simple.map and usr/share/hwdata/pci.ids) and copy them into oem.tgz in the same directory structure. The extracted oem.tgz archive gets overlaid over the extracted binmod.tgz so the paths need to be identical. In the pci.ids file I had to add entries for my ICH controller...I added all of the entries based on the various BIOS configurations (the top and bottom lines are from the original file and can be used as a reference point):

27df 82801G (ICH7 Family) IDE Controller
2820 82801H (ICH8 Family) 4 port SATA IDE Controller
2821 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA AHCI Controller
2824 82801HB (ICH8) 4 port SATA AHCI Controller
2825 82801H (ICH8 Family) 2 port SATA IDE Controller
2828 Mobile SATA Controller IDE (ICH8M)

I then had to update the the simple.map to include the directives to load the appropriate kernel drivers for these devices...

8086:2820 0000:0000 storage ata_piix
8086:2821 0000:0000 storage ahci
8086:2824 0000:0000 storage ahci
8086:2825 0000:0000 storage ata_piix
8086:2828 0000:0000 storage ata_piix

With these updates tar-gziped into oem.tgz on my USB stick and configing my BIOS to use AHCI I was able to install ESXi. I did get a kernel panic though as my update was not inside the installed image. Inside the install.tgz archive there is a VMware-VMvisor-big-3.5.0_Update_2-103909.i386.dd.bz2 which I needed to extract, mount, update with the same changes, unmount compress, and re-archive. The VMware-VMvisor-big-3.5.0_Update_2-103909.i386.dd.bz2 is really just a harddrive disk dump and the oem.tgz file lives on partition 5 inside this image. I used ISCSI with fileio to mount the image, but that is because I already had ISCSI all set up. (I have added instructions for an easier method than iSCSI...see my comment at the bottom or just go to http://www.grid.org/blog/cameron/updating-vmware-esxi-disk-dump-file)

After I updated install.tgz and put that back on my USB stick I was able to install and run vmware ESXi. I have been using it now for about 2 weeks and am loving the snapshot manager, increased network configuration options, and general performance improvement. Of course when the recent Time Bomb issue happened the VMWare patch broke my install. If you wish to apply the new VMWare patch you must:

1. Before installing the patch use the VMWare SSH hack that you can find on the internet to enable ssh access.
2. Reboot your host
3. Apply the patch
4. Ssh to the host
5. Run 'find / -name oem.tgz' and note the locations of all of the oem.tgz files.
6. Scp the oem.tgz from your USB stick to the host.
7. Copy oem.tgz to all of the locations displayed in 5.
8. Reboot your machine

After running this I was able to have a Time Bomb patched VMWare ESXi server running on my ICH8 board.

-Cameron