Recently, I was approached if I could help out installing VMWare ESXi on a Hetzner server. The server had already been rented and the LARA console showed a sad error message stating the ESXi could not find any network cards. D’uh. Bummer that :)
Reading the Hetzner Wiki concerning this issue it does go into great detail on how to configure the ESXi once it is up and running but there is a bullet point which can be easily overlooked:
- The installation of vSphere 5.5 on all other models requires an extension of the installation media to include drivers for Realtek network cards or for PX60/70 an update of the igb driver.
The standard VMware-VMvisor-Installer-5.5.0-1331820.x86_64.iso used to install ESXi on the Hetzner machine did not have the updated igb drivers resulting in the failure to find/see the network card.
VMWare does provide a simple and easy way to slipsteam addition drivers thus creating a custom CD image but it requires a Windows with Powershell and the installation of VMware-PowerCLI-5.5.0-1295336.exe. At this stage it is a good idea to also install VMware-viclient-all-5.5.0-1281650.exe since you should test the access to the newly installed ESXi host using the VSphere client.
The actual updated igb driver package can be found here:igb-220.127.116.11-1290233.zip. Extract the contents of the archive on the desktop which should produce the following file igb-18.104.22.168-offline_bundle-1290233.zip. Once the installation is complete a new icon should appear on the desktop: VMware vSphere PowerCLI. Double-Click on the icon to open a Powershell containing the VMWare cmdlets. Navigate to the desktop folder.The following script assumes the archive to be on the desktop. Do not worry if the first and the last command seem to take a very long time. Especially the last command. Just be patient.
# Add VMware Online depot Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml # Clone the ESXi 5.5 GA profile into a custom profile $CloneIP = Get-EsxImageProfile ESXi-5.5.0-1331820-standard $MyProfile = New-EsxImageProfile -CloneProfile $CloneIP -Vendor $CloneIP.Vendor -Name (($CloneIP.Name) + "-customized") -Description $CloneIP.Description # Add latest versions of missing driver packages to the custom profile Add-EsxSoftwareDepot ./igb-22.214.171.124-offline_bundle-1290233.zip # Export the custom profile into ISO file, this may take a while Export-EsxImageProfile -ImageProfile $MyProfile -ExportToISO -FilePath ./ESXi-5.5.0-1331820-standard-customized.iso
This should result on a new ISO image with the file name ESXi-5.5.0-1331820-standard-customized.iso on the desktop. Use this to install ESXi on a Hetzner server with the help of the LARA console. Just follow the installation instructions.
Once the ESXi server is up and running, try to connect to it using the VSphere client. In addition, now would be a good point to install any updates that may exist. In this case, VMWare has an update available ESXi550-201312001.zip. Download the archive and copy the archive to the ESXi server (either by using scp or the VSphere client). Log into the ESXi server using ssh and change into the directory where the update archive is located. Then issue the following commands:
vim-cmd /hostsvc/maintenance_mode_enter esxcli software vib update --depot=$(pwd)/ESXi550-201312001.zip vim-cmd /hostsvc/maintenance_mode_exit reboot
This will take a while and the machine should reboot. After that, you should have a working and updated ESXi server running on your Hetzner server. What you do next is up to you :)
You can set the security settings of the ESXi via the VSphere client (be careful not to lock yourself out) but you can make those changes on the command line too. Log into the ESXi machine using ssh and execute the following command:
esxcli network firewall ruleset list
Securing a service, for example webAccess, from the command line is not that difficult. You can disable a service or only allow access from specific IP addresses.
- Disable a service:
esxcli network firewall ruleset set --ruleset-id webAccess --enabled false
- Prepare the IP address list by disabling the allowed-all flag
esxcli network firewall ruleset set --ruleset-id webAccess --allowed-all false
- Setting the specific IP addresses allowed to access the service
esxcli network firewall ruleset allowedip list esxcli network firewall ruleset allowedip add --ruleset-id webAccess --ip-address 126.96.36.199 esxcli network firewall ruleset allowedip add --ruleset-id webAccess --ip-address 188.8.131.52 esxcli network firewall ruleset allowedip add --ruleset-id webAccess --ip-address 184.108.40.206 esxcli network firewall ruleset allowedip list
- To remove the IP address(es) from the list, use the keyword remove instead of add.
Updating the ESXi host once virtual machines are running
- Obtain a list of running virtual machines
esxcli vm process list
- Shutting down the virtual machine (prefer if it has vmware-tools running)
esxcli vm process kill --type=[soft,hard,force] --world-id=WorldNumber
- Shutting down the virtual machine if the guest operating system supports the ACPI shutdown event
vim-cmd vmsvc/getallvms vim-cmd vmsvc/power.getstate Vmid vim-cmd vmsvc/power.shutdown Vmid
- If it fails using the ACPI shutdown event trigger
vim-cmd vmsvc/power.off Vmid
- Follow the steps outlined above on how to update (set maintenance mode, update, exit maintenance mode)
- Starting a virtual machine
vim-cmd vmsvc/getallvms vim-cmd vmsvc/power.on Vmid
The slipstreaming is no longer neccessary with 5.5.0U1 — thecurrent IGB package is included now. Unfortunately the system (PX60) is unstable. As long as it is idle, no VMs running, everything is fine. When I start any VM, e. g., simply a Live Linux, it lasts about 1–3 hours until I get a “pink screen of death” complaing about a nonmaskable interrupt. I detailed hardware was conducted — no findings. The Hetzner forum had a story about a similar situation with the PX70 w/o solution. Same with 5.1U1 and another, external NIC (Intel e1000). Return the box and order a new one, maybe a PX90 which has another mainboard? What is your experience?
So there are problems with the U1 update package!
I did suspect something was not right because when I transferred an ISO image to a datastore using the VSphere client from within a virtual machine running on the ESXi machine it simply crashed out. A reboot request issued via the Hetzner portal quickly resolved the problem but the second try the same thing happend (so I can reproduce the problem but the server is already “in production” so further testing is delayed for now until things really get worse and another reproducable condition is found which will crash the complete ESXi server.
For the time being I just scp the ISO images to the datastore so not being able to do so via the VSphere client is not much trouble for me.
Still, scary that you can bring down the whole ESXi host by doing something simple as transferring data to a datastore… :/ The machines running the ESXi are just a Linux, some Windows and an OpenBSD, so I will have to see what will crash the ESXi next :)
Concerning the igb device driver, I just hope the article can still be helpful to people needing to slipstream other device drivers into the official ISO image. Thank you nevertheless for pointing out the updated igb driver is contained in the 5.5.0U1!
This will solve your problem most likely.