Felipe Alfaro Solana

Let’s face it: installing VMware ESXi from a CD-ROM or from a USB key is painfully slow. Installing from the network is faster and more flexible. And preparing VMware to be installed from PXE turned out to be very easy.

The ISC DHCP configuration file could look like this:

# cat /etc/dhcpd.conf
default-lease-time 86400;
max-lease-time 604800;
option subnet-mask 255.255.255.0;
option broadcast-address 1.0.0.255;
option domain-name-servers 1.0.0.1;
option domain-name "example.com";

subnet 1.0.0.0 netmask 255.255.255.0 {
        range 1.0.0.100 1.0.0.254;
        option routers 1.0.0.2;
        option ntp-servers 1.0.0.3;
}

host esx {
        hardware ethernet 00:aa:bb:cc:dd:ee;
        fixed-address esx;
        next-server 1.0.0.4;
        filename "pxelinux.0";
}

The important bits are in the host esx section, where PXE boot support is enabled by means of the next-server and the filename directive. next-server specifies the IP address (or DNS name) of the TFTP server to be used to download the PXE boot loader and filename the file that stores the PXE boot loader code.

Looking inside the TFTP server, we can see that the tftpboot root directory is very simple: it consists of a standard pxelinux.0 PXE boot loader, a pxelinux.cfg directory where the configuration files are stored and a directory for all VMware-related files. pxelinux.0 is just part of the syslinux project. pxelinux.cfg has to be created by hand. vmware-esxi-4-0-0 contains files copied directly from the VMware ESXi 4 installable ISO image:

# ls -l /tftpboot
total 40
-rw-r--r--  1 root  wheel  14776 Sep 18 03:17 pxelinux.0
drwxr-xr-x  2 root  wheel    512 Sep 18 03:43 pxelinux.cfg
drwxr-xr-x  2 root  wheel    512 Sep 18 03:50 vmware-esxi-4-0-0

For all different naming options for configuration files stored under pxelinux.cfg, check the manual page for pxelinux or search the Internet. In my case, I just chose 01-${MAC} where ${MAC} is the MAC address of the Ethernet interface used to PXE-boot the machine where ESXi is to be installed. In this case, ${MAC} is 00-aa-bb-cc-dd-ee.

The contents of the configuration file are in fact a slightly modified copy of the contents of the isolinux.cfg file from the VMware ESXi 4.0 installable ISO image. The only differences are the default and label directives and the adjusted path names for the kernel and modules: all these files live inside their own directory to avoid polluting the tftpboot root.

# cat /tftpboot/pxelinux.cfg/01-00-aa-bb-cc-dd-ee
default esxi
label esxi
kernel vmware-esxi-4-0-0/mboot.c32
append vmware-esxi-4-0-0/vmkboot.gz
   --- vmware-esxi-4-0-0/vmkernel.gz
   --- vmware-esxi-4-0-0/sys.vgz
   --- vmware-esxi-4-0-0/cim.vgz
   --- vmware-esxi-4-0-0/ienviron.tgz
   --- vmware-esxi-4-0-0/image.tgz
   --- vmware-esxi-4-0-0/install.tgz

The files stored inside the vmware-esxi-4-0-0 directory were copied directly from the VMware ESXi 4.0 installable ISO image, as mentioned above:

# ls -l /tftpboot/vmware-esxi-4-0-0
total 694704
-r--r--r--  1 root  wheel   12730046 Sep 18 03:15 cim.vgz
-r--r--r--  1 root  wheel    5818848 Sep 18 03:15 ienviron.tgz
-r--r--r--  1 root  wheel  288629638 Sep 18 03:17 image.tgz
-r--r--r--  1 root  wheel      21456 Sep 18 03:17 install.tgz
-r-xr-xr-x  1 root  wheel      47404 Sep 18 03:44 mboot.c32
-r--r--r--  1 root  wheel   46184258 Sep 18 03:15 sys.vgz
-r--r--r--  1 root  wheel      16805 Sep 18 03:15 vmkboot.gz
-r--r--r--  1 root  wheel    2044368 Sep 18 03:15 vmkernel.gz

When it is not possible to install VMware ESXi 4.0 from a CD/DVD drive, and if the machine supports booting from USB, one can easily install from a USB drive. Preparing the USB drive to install ESXi 4.0 from it is very easy:

Create a FAT32 partition on the USB drive:

# install-mbr /dev/sdX
# fdisk /dev/sdX
...
# mkfs.vfat /dev/sdX1

Make sure the FAT32 partition is tagged as bootable/active in the MBR and that preferably it has a valid Win32 FAT32 partition type.

Next, copy the contents of the ESXi 4.0 CD into the FAT32 partition from the USB drive:

# mount -o loop /path/to/VMware-VMvisor-Installer-4.0.0-171294.x86_64.iso /mnt
# mount /dev/sdX1 /media
# cp /mnt/* /media
# mv /media/isolinux.cfg /media/syslinux.cfg
# umount /media
# umount /mnt

The last step consists of installing syslinux into the FAT32 partition:

# syslinux -s /dev/sdX1

Done!

On this second post I want to talk about the interaction problems I experienced with the HP SmartArray P212 controller in this computer. The HP SmartArray P212 controller is certified for VMware ESXi 4.0 and Solaris 10. Initially I thought that using VMware would be useful to me in order to play with Solaris and even Windows 7.

However, I haven’t been able to get VMware ESXi 4.0 to work properly on this controller. If I create 4 logical drives in the HP controller, one for each phyisical disk, VMware finds the drives and figures out their right sizes. However, if configure a 3-drive RAID-5 logical volume in the HP controller, yielding a usable 3.0TB volume size, VMware finds and reports a 0.0B-sized volume. I tried different options from the HP SmartArray BIOS, like limiting the maximum bootable partition size, but the end result is always the same: VMware sees a 0.0B logical volume that can’t be used to install VMware neither to store virtual disks.

In the end, I ditched VMware ESXi 4.0 in favor of OpenSolaris, at least on this machine. I could have created 4 logical volumes, but it doesn’t make much sense for VMware itself. It makes perfect sense when running Solaris and using RAIDZ, though.

I haven’t been able to found any explanation to this problem other than VMware does not support LUNs bigger than 2TB. Is this the case? Do any of you have experience with VMware and LUNs larger than 2TB?

It seems that the oust of Diane Greene is having big consequences. One of them is that VMware ESXi is now a free product. I see this like a direct attack to Microsoft’s attempt to get into the virtualization market. What does this mean? That there is little reason to choose Microsoft’s hypervisor when you can choose a more mature one that is now free.

Of course, hardware requirements for ESXi still make it hard to afford for small companies, but these are very good news in any case.

The hypervisors war is getting very hot!

I have been fighting for quite some time to get Solaris 10 installed and working properly under VMwae Fusion 1.1 on an Intel Core 2 Duo Mac. I think this is an interaction problem between VMware Fusion and Solaris 10. Even if you tell VMware Fusion that you want to install Solaris in 32-bit mode, VMware doesn’t disable the 64-bit, long-word instruction support (available from the host). Thus, Solaris 10 is able to detect that 64-bit, long-word instructions are available and boots a 64-bit kernel. VMware then complains about the fact that the virtual machine was configured in 32-bit mode but the guest is trying to execute 64-bit instructions.

The work-around is pretty easy: it consists of disabling 64-bit, long-word instructions in the VMware configuration file for the guest. This is described in greater detail in article Installing Solaris 10 as a 32-Bit Guest Operating System on a 64-Bit Host Machine but consists mainly in editing the .vmx virtual machine’s configuration file and adding the following line:

monitor_control.disable_longmode = 1

Rebooting the installation, or booting an already installed virtual machine system should make Solaris boot into 32-bit mode.

From How to improve disk I/O performances with VMware Workstation 5:

Memory trimming

Workstation checks which part of the guest OS virtual memory is not used and allocates it back to the host OS. This permits to have more concurrent virtual machines running but everytime the guest OS asks back for its memory it suffers a performance degradation.

So, if you have enough free RAM for all planned concurrent VMs, be sure to disable memory trimming for guest OSes adding the following line to the virtual machine configuration (.vmx) file:

MemTrimRate=0

Page sharing

VMware uses a page sharing technique to allow guest memory pages with identical contents to be stored as a single copy-on-write page. Page sharing decreases host memory usage, but consumes system resources, potentially including I/O bandwidth.

You may want to avoid this overhead for guests for which host memory is plentiful and I/O latency is important. To disable page sharing, add the following line to the virtual machine configuration (.vmx) file:

sched.mem.pshare.enable=FALSE option