VMkernel
What is VMkernel?
VMkernel is a POSIX (Portable Operating System Interface)-like operating system (OS) developed by VMware. It acts as a liaison between virtual machines (VMs) and the physical hardware that supports them, such as ESXi hosts. A VM needs VMkernel to communicate with the ESXi server.
The functionalities of VMkernel
VMware calls VMkernel a microkernel because it runs on bare metal, or directly on VMware ESX hosts. It includes many functions that are also available in other OSes, such as process creation, process control, process threads, signals, file system, etc.
VMkernel is designed to support running multiple VMs. Some of its core functions include:
- device drivers
- resource scheduling
- I/O stacks
It is responsible for allocating resources -- such as memory, storage and CPU -- from the host to the VM, and it schedules CPUs, provides hardware abstraction and other OS services. The VMkernel also controls several special services, such as:
- vMotion
- fault tolerance
- traffic management
- Network File System (NFS)
- iSCSI
VMkernel file system
VMkernel uses a simple in-memory file system to hold the ESXi configuration files, log files and staged patches. It is independent of the VMware Virtual Machine File System (VMFS) where VMs are stored. The system's design is similar to the service console of ESX, where remote command line interfaces (CLIs) provide file management capabilities.
HTTPS get and put commands are used to access to the file system. Users and groups are configured locally on the server, and are controlled by local privileges for authentication and access.
The log files in the file system cannot survive a reboot after the system is shut down. However, users can save all log information on an external system since ESXi can configure a remote syslog server.
VMkernel users and groups
As with other OSes, groups can be used in VMkernel to combine multiple users or to set privileges. Groups provide a way to differentiate between users accessing the system via the Virtual Infrastructure Client or the Virtualization Infrastructure Manager (VIM) API.
However, administrative privileges can -- and should -- be set individually for each user/group. Passwords for users and groups are generated using standard crypt functions.
VMkernel user worlds
A process running in VMkernel is referred to as a "user world." It runs in a limited environment compared to other general-purpose POSIX-compliant OSes like Linux. These limitations include:
- the set of available signals
- the system API is a subset of POSIX
- a limited /proc file system
- a single swap file for all user world processes
In VMkernel, a user world is not intended to run arbitrary applications. Rather, it provides a framework for processes or native VMkernel applications to run in the hypervisor environment. Some of these processes are:
- the hosted process to authenticate users and track user privileges
- the vpxa process to connect to VirtualCenter
- a syslog daemon that forwards all logs to a remote target
- a process to manage the initial discovery of an iSCSI target
- a process to enable NTP-based time synchronization
- a process for SNMP monitoring
VMkernel networking layer
The VMkernel networking layer provides connectivity to hosts and handles standard system traffic of vSphere vMotion, IP storage, fault tolerance, etc. The TCP/IP stacks at the VMkernel level include:
Default TCP/IP stack: It provides networking support for managing traffic between the vCenter Server and ESXi hosts, and for system traffic like IP storage traffic and fault tolerance.
vMotion TCP/IP stack: This stack supports traffic for live migration of VMs, and it provides better isolation for vMotion traffic.
Provisioning TCP/IP stack: This stack supports traffic for VM cold migration, snapshot migration and cloning. It can also be used to handle Network File Copy (NFC) traffic during long-distance vMotion. ESXi uses NFC to copy and move data between data stores.
Custom TCP/IP stacks: In addition to the above stacks, users can add custom TCP/IP stacks to handle networking traffic for custom applications.
VMkernel adapters and system traffic types: A dedicated and separate VMkernel adapter can be used for every traffic type.
Important system traffic types include:
Management traffic: By default, when the ESXi software is installed, a vSphere standard switch is created on the host with the VMkernel adapter for management traffic. The management traffic carries the configuration and management communication for ESXi hosts, vCenter Server and host-to-host high availability traffic.
vMotion traffic: As the name suggests, vMotion traffic (unencrypted) accommodates vMotion. A VMkernel adapter for vMotion is required both on the source and target hosts. Multiple NICs can be used for vMotion to increase bandwidth.
Provisioning traffic: It carries the data transferred for VM cold migration, snapshot migration and cloning.
IP storage traffic: It handles the connection for storage types, including software iSCSI, dependent-hardware iSCSI and NFS that use standard TCP/IP networks.
Fault tolerance traffic: This traffic involves the data that is sent from the primary fault-tolerant VM to the secondary fault-tolerant VM. This data transfer typically happens over the VMkernel networking layer.
In addition to the above traffic types, a VMkernel can also accommodate these traffic types:
See also: How do you reconfigure access to a VMkernel port?, Creating a VMkernel connection to optimize ESXi storage traffic and Change VMkernel port MTU size to reduce packet fragmentation. Explore: Use iSCSI storage for VMware to create a data store, How do you prevent host isolation during network maintenance? and Get familiar with the basics of vMotion live migration.
