ESXi Hosts Health Check Handbook

1. Overview & Purpose

This handbook provides a complete, step-by-step health check procedure for ESXi 8.0 hosts deployed within a VCF 9.0 environment. Use this during:

• Routine maintenance — Weekly/monthly proactive host verification
• Pre/post-upgrade — Before and after ESXi patches or VCF lifecycle operations
• Incident response — When VMs are experiencing performance or connectivity issues
• Capacity planning — Evaluating host resource utilization and VM density

Scope

2. Prerequisites

Required Access

Enable SSH on ESXi (if disabled)

# Via DCUI: F2 → Troubleshooting Options → Enable SSH
# Via PowerCLI:
Get-VMHost -Name <esxi-host> | Get-VMHostService | Where {$_.Key -eq "TSM-SSH"} | Start-VMHostService
# Via esxcli (from vCenter or DCUI):
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh

Environment Variables

# Set per-host or iterate across all hosts
export ESXI_HOSTS="192.168.1.201 192.168.1.202 192.168.1.203 192.168.1.204"
export ESXI_USER="root"
export ESXI_PASS="YourPassword123!"

# Loop template for multi-host checks
for HOST in $ESXI_HOSTS; do
  echo "======== $HOST ========"
  ssh root@$HOST '<command>'
done

3. Quick Reference — All Checks Summary

4. Hardware Health

4.1 Hardware Sensors (IPMI)

What: Read IPMI sensor data to verify temperatures, voltages, fan speeds, and power supply status.

Why: Hardware sensor warnings precede failures. Catching them early prevents unplanned outages.

CLI Method

esxcli hardware ipmi sdr list

Expected Output (Healthy):

Name                    Entity ID  Sensor Type     Reading  Units    Status
---------------------------------------------------------------------------
Inlet Temp              0x01.0x01  Temperature     22       C        0x01
Exhaust Temp            0x01.0x02  Temperature     35       C        0x01
CPU1 Temp               0x03.0x01  Temperature     48       C        0x01
CPU2 Temp               0x03.0x02  Temperature     45       C        0x01
FAN1                    0x1d.0x01  Fan             6200     RPM      0x01
FAN2                    0x1d.0x02  Fan             6100     RPM      0x01
P1-DIMMA1 Temp          0x20.0x01  Temperature     32       C        0x01
PSU1 Status             0x0a.0x01  Power Supply    —        —        0x01
PSU2 Status             0x0a.0x02  Power Supply    —        —        0x01

Status Codes

Pass / Warn / Fail

Additional Sensor Commands

# Detailed hardware platform info
esxcli hardware platform get

# BMC/IPMI firmware info
esxcli hardware ipmi bmc get

4.2 Memory Hardware Errors

What: Check for correctable (CE) and uncorrectable (UE) memory errors.

# Memory overview
esxcli hardware memory get

# Check DIMM status
esxcli hardware memory dimm list

Expected Output:

Physical Memory: 549,755,813,888 Bytes (512 GB)
Reliable Memory: 549,755,813,888 Bytes

DIMM Locator     Bank     Type   Speed   Size      Status
----------------------------------------------------------
P1-DIMMA1        Bank 0   DDR5   4800    32768 MB  ok
P1-DIMMB1        Bank 1   DDR5   4800    32768 MB  ok
P1-DIMMC1        Bank 2   DDR5   4800    32768 MB  ok
...

Check VMkernel Log for Memory Errors

grep -i "machine check\|memory error\|ECC\|CECC\|UECC" /var/log/vmkernel.log | tail -20

4.3 CPU Health

# CPU info
esxcli hardware cpu list | head -30

# CPU global stats
esxcli hardware cpu global get

Expected Output:

CPU Packages: 2
CPU Cores: 32
CPU Threads: 64
Hyperthreading Active: true
Hyperthreading Supported: true

4.4 PCI Devices

# List all PCI devices
esxcli hardware pci list | grep -E "Device Name|Vendor Name|Address"

4.5 Firmware Versions

# BIOS info
esxcli hardware platform get

# Storage controller firmware
esxcli storage core adapter list

5. Storage Health

5.1 VMFS Datastores

What: Verify all VMFS datastores are mounted, accessible, and have adequate free space.

# List VMFS extents
esxcli storage vmfs extent list

Expected Output:

Volume Name  VMFS UUID                             Extent #  Device Name                           Partition
-----------------------------------------------------------------------------------------------------------------
datastore1   61234567-abcdef01-2345-001122334455   0         naa.600508b4001234567890abcdef012345   3
vsanDatastore 71234567-abcdef01-2345-001122334455  0         —                                     —

Check Datastore Free Space

# Datastore capacity (via df equivalent)
esxcli storage filesystem list

Expected Output:

Mount Point              Volume Name    UUID                                  Mounted  Type  Size            Free
------------------------------------------------------------------------------------------------------------------
/vmfs/volumes/ds1        datastore1     61234567-abcdef01...                  true     VMFS-6  1099511627776  549755813888
/vmfs/volumes/vsanDs     vsanDatastore  71234567-abcdef01...                  true     vsan    4398046511104  2199023255552

5.2 SCSI Device Status

# List all SCSI devices
esxcli storage core device list

# Check for devices in APD/PDL state
esxcli storage core device list | grep -E "Display Name|Status|Is Perennially Reserved"

Key Indicators:

5.3 HBA Health & Multipathing

What: Verify all storage paths are active and HBAs are functioning.

# List HBAs
esxcli storage core adapter list

Expected Output:

HBA Name  Driver       Link State  UID                            Description
------------------------------------------------------------------------------
vmhba0    lsi_mr3      link-up     sas.5001636001234567           LSI Logic SAS3108
vmhba1    lsi_mr3      link-up     sas.5001636001234568           LSI Logic SAS3108
vmhba32   iscsi_vmk    online      iqn.1998-01.com.vmware:esxi01  VMware iSCSI

Multipathing Status

# List all paths and their state
esxcli storage nmp path list

Expected Output:

Runtime Name: vmhba0:C0:T0:L0
  Device: naa.600508b400123456...
  Adapter: vmhba0
  LUN: 0
  State: active
  Transport: sas

Runtime Name: vmhba1:C0:T0:L0
  Device: naa.600508b400123456...
  Adapter: vmhba1
  LUN: 0
  State: active
  Transport: sas

5.4 NFS Mounts

# List NFS datastores
esxcli storage nfs list

Expected Output:

Volume Name  Host            Share       Accessible  Mounted  Read-Only  Hardware Acceleration
----------------------------------------------------------------------------------------------
nfs-backup   192.168.1.100   /exports   true        true     false      Supported

5.5 Disk Latency & SMART

What: Check real-time disk latency and SMART health data.

Check Latency via esxtop

# Interactive mode — press 'd' for disk view
esxtop

# Batch mode (3 samples, 5-second intervals)
esxtop -b -d 5 -n 3 | grep -E "DAVG|KAVG|GAVG"

Key Metrics:

SMART Data (Local Disks)

# Check SMART status for local disks
esxcli storage core device smart get -d <device-naa-id>

6. Networking Health

6.1 VMkernel Adapters

What: Verify all VMkernel adapters have correct IPs, MTU, and enabled services.

# List VMkernel interfaces
esxcli network ip interface list

Expected Output:

Name   MAC Address        Enabled  MTU   Portgroup/DVPort    Stack       IPv4 Address
---------------------------------------------------------------------------------------
vmk0   00:50:56:01:aa:01  true     1500  Management Network  defaultTcpipStack  192.168.1.201
vmk1   00:50:56:01:aa:02  true     9000  vMotion             vmotion     192.168.10.201
vmk2   00:50:56:01:aa:03  true     9000  vSAN                defaultTcpipStack  192.168.12.74
vmk10  00:50:56:01:aa:04  true     1600  nsx-overlay         defaultTcpipStack  192.168.14.201

Check VMkernel Tagged Services

esxcli network ip interface tag get -i vmk0
esxcli network ip interface tag get -i vmk1
esxcli network ip interface tag get -i vmk2

6.2 vSwitch / vDS Configuration

# List standard vSwitches
esxcli network vswitch standard list

# List Distributed vSwitches
esxcli network vswitch dvs vmware list

Expected Output (DVS):

Name: DSwitch-Management
  Configured Ports: 2048
  Max Ports: 2048
  MTU: 9000
  CDP Status: both
  Uplinks: vmnic0, vmnic1

Name: DSwitch-Compute
  Configured Ports: 2048
  Max Ports: 2048
  MTU: 9000
  Uplinks: vmnic2, vmnic3

6.3 Physical Uplink Status

What: Verify all physical NICs are connected, at expected speed, and link is up.

esxcli network nic list

Expected Output:

Name    PCI Device  Driver  Admin Status  Link Status  Speed  Duplex  MTU    MAC Address
-------------------------------------------------------------------------------------------
vmnic0  0000:3b:00  i40en   Up            Up           25000  Full    9000   00:50:56:01:aa:10
vmnic1  0000:3b:01  i40en   Up            Up           25000  Full    9000   00:50:56:01:aa:11
vmnic2  0000:5e:00  i40en   Up            Up           25000  Full    9000   00:50:56:01:aa:12
vmnic3  0000:5e:01  i40en   Up            Up           25000  Full    9000   00:50:56:01:aa:13

6.4 NIC Driver & Firmware

# Driver details per NIC
esxcli network nic get -n vmnic0

Expected Output:

Advertised Auto Negotiation: true
Auto Negotiation: true
Driver Info:
  Bus Info: 0000:3b:00.0
  Driver: i40en
  Firmware Version: 9.20
  Version: 2.5.3.0
Link Detected: true
Link Status: Up

6.5 CDP / LLDP Neighbor Info

What: Discover the physical switch port each ESXi NIC is connected to.

# CDP (Cisco Discovery Protocol)
esxcli network vswitch dvs vmware lacp config get
# or via vim-cmd:
vim-cmd hostsvc/net/query_networkhint --pnic-name=vmnic0

Using Python for CDP/LLDP

# Python one-liner to get CDP info
python3 -c "
from pyVim.connect import SmartConnect
import ssl
# ... (connect to host and query PhysicalNicHintInfo)
"

6.6 vmkping Connectivity Tests

# Management network
vmkping -I vmk0 <vcenter-ip>

# vMotion network (jumbo frame test)
vmkping -I vmk1 -d -s 8972 <other-host-vmotion-ip>

# vSAN network (jumbo frame test)
vmkping -I vmk2 -d -s 8972 <other-host-vsan-ip>

# NSX TEP (MTU 1600)
vmkping -I vmk10 -d -s 1572 <other-host-tep-ip>

7. Services Health

7.1 Service Listing

# List all services and their status
esxcli system process list | head -60

# Alternative — list via chkconfig
chkconfig --list | sort

7.2 Critical Services

Verify Critical Services

/etc/init.d/hostd status
/etc/init.d/vpxa status
/etc/init.d/fdm status
/etc/init.d/ntpd status
/etc/init.d/nsx-mpa status
/etc/init.d/nsx-proxy status

Expected Output for each:

hostd is running.

8. NTP Configuration

What: Verify NTP is configured, running, and the host clock is synchronized.

# Check NTP configuration
esxcli system ntp get

Expected Output:

Enabled: true
Loglevel: warning
Server:
  - 192.168.1.1
  - 192.168.1.2

# Check NTP service status
/etc/init.d/ntpd status

# Check time offset
esxcli system time get
ntpq -p

Expected ntpq Output:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*192.168.1.1     .GPS.            1 u   64  128  377    0.543    0.125   0.043
+192.168.1.2     .GPS.            1 u   32  128  377    0.621    0.213   0.051

9. Syslog Configuration

What: Verify syslog is forwarding to a remote collector.

# Get syslog config
esxcli system syslog config get

Expected Output:

Default Network Retry Timeout: 180
Dropped Log File Rotation Size: 100
Dropped Log File Rotations: 10
Log Output: /scratch/log
Log To Unique Subdirectory: false
Remote Host: udp://loginsight.lab.local:514

# Check syslog service
esxcli system syslog mark --message="Health Check Test $(date)"
# Then verify it appears on remote collector

10. Scratch Partition

What: Verify the scratch partition points to persistent storage (not ramdisk).

vim-cmd hostsvc/advopt/get ScratchConfig.ConfiguredScratchLocation
vim-cmd hostsvc/advopt/get ScratchConfig.CurrentScratchLocation

Expected Output:

ConfiguredScratchLocation: /vmfs/volumes/datastore1/.locker-<hostname>
CurrentScratchLocation: /vmfs/volumes/datastore1/.locker-<hostname>

11. Core Dump Configuration

# Check core dump partition
esxcli system coredump partition get

# Check network core dump
esxcli system coredump network get

Expected Output:

Active: true
Configured: true
Partition: naa.xxx:7

# Network dump:
Enabled: true
Host VmkNic: vmk0
Network Server IP: 192.168.1.50
Network Server Port: 6500

12. Security Health

12.1 SSH Status

# Check if SSH is running
/etc/init.d/SSH status

# Check SSH timeout
esxcli system settings advanced list -o /UserVars/ESXiShellInteractiveTimeOut
esxcli system settings advanced list -o /UserVars/ESXiShellTimeOut

12.2 Lockdown Mode

vim-cmd hostsvc/hostsummary | grep lockdownMode

Expected Output:

lockdownMode = "lockdownNormal"

12.3 ESXi Firewall Rules

# List all firewall rulesets
esxcli network firewall ruleset list

# Check specific rules
esxcli network firewall ruleset rule list --ruleset-id=sshServer

Key Rulesets to Verify

12.4 Certificate Validity

# Check the ESXi host certificate
openssl x509 -in /etc/vmware/ssl/rui.crt -noout -dates -subject

Expected Output:

notBefore=Jan 15 00:00:00 2026 GMT
notAfter=Jan 15 00:00:00 2028 GMT
subject=CN = esxi-01.lab.local, ...

12.5 Account Lockout Policy

esxcli system account policy get

Expected Output:

Maximum Failed Login Attempts: 5
Unlock Time (seconds): 900

13. Performance Health

13.1 CPU Ready & Co-Stop

What: Check if VMs are experiencing CPU scheduling delays (CPU ready time) or NUMA/co-stop issues.

esxtop Method

# Interactive — press 'c' for CPU view
esxtop

# Batch mode
esxtop -b -d 5 -n 3 > /tmp/esxtop-cpu.csv

Key Columns in CPU View:

13.2 Memory Ballooning & Swap

# Host memory summary
esxcli hardware memory get

# Check ballooning and swap via esxtop ('m' for memory view)
esxtop

Key Memory Metrics:

13.3 Host Overcommit Ratio

# Calculate overcommit ratio
# Total VM vCPUs / Physical threads
esxcli hardware cpu global get
vim-cmd vmsvc/getallvms | wc -l

# Total configured VM memory vs physical memory
esxcli hardware memory get

Recommended Ratios

14. Boot Configuration

What: Verify the boot device, boot banks, and acceptance level.

# Boot device
esxcli system boot device get

# Boot bank status
/bin/bootbank-util status

# Acceptance level
esxcli software acceptance get

Expected Output:

Boot Device: mpx.vmhba0:C0:T0:L0
Boot Filesystem UUID: 61234567-abcdef01...
Boot bank: /bootbank (valid)
Alt boot bank: /altbootbank (valid)
Acceptance Level: VMwareCertified

15. Patch / VIB Level

What: Verify the installed ESXi image profile matches the VCF BOM.

# Installed image profile
esxcli software profile get

Expected Output:

Name: (Updated) ESXi-8.0U3-12345678-standard
Vendor: VMware, Inc.
Creation Time: 2026-01-15T00:00:00
Modification Time: 2026-01-15T00:00:00
Stateless Ready: True

# List all installed VIBs
esxcli software vib list | head -30

# Check for specific VIB
esxcli software vib list | grep -i nsx

VCF 9.0 Expected Versions (check BOM)

16. Port Reference Table

Inbound Ports (to ESXi)

Outbound Ports (from ESXi)

17. Common Issues & Remediation

17.1 PSOD (Purple Screen of Death)

17.2 Storage APD / PDL

17.3 Network Partition

17.4 Host Disconnection from vCenter

# On ESXi host — restart management agents
/sbin/services.sh restart

# If that doesn't work, restart only vpxa
/etc/init.d/vpxa restart

# Check hostd logs for errors
tail -100 /var/log/hostd.log | grep -i error
tail -100 /var/log/vpxa.log | grep -i error

17.5 Performance Degradation

18. CLI Quick Reference Card

System Information

Storage Commands

Networking Commands

Service Commands

Security Commands

Software / Patch Commands

Performance / Diagnostic Commands