ltstudiooo - Fotolia
KVM troubleshooting in six simple steps
To troubleshoot common KVM issues, start by checking the logs. Then, isolate the issue and check the VM status using virsh commands. Finally, check the disk space, memory and networking.
KVM is a well-known hypervisor that is free to use and works well. However, being able to do some basic KVM troubleshooting is a key skill that will help when things go a bit awry.
As any administrator knows, logs are vital to effective troubleshooting. Within KVM there are two types of logs: hypervisor logs -- KVM, in this case -- and VM logs. Depending on your configuration and OS of choice, they might differ, but KVM logs, such as the messages file, can be found under the /var/logs folder. These logs are associated with the KVM service, not VMs.
Each VM has its own log file, and these are located under /var/log/libvirt/qemu. These hold the information that pertains only to the VM in question, such as startup, shutdown and VM hardware issues.
Isolate the issue
The next step in KVM troubleshooting is to isolate the issue. Are all your KVM machines affected or is it just one? If all of them are affected, the problem might be with the KVM service rather than the VMs.
Check the service by first being root and then using the following command:
systemctl status libvirtd
The important line is the active line -- it should show Active: active -- alongside the uptime on the right side of the screen, as shown in Figure A below.
If there are errors, you can restart the service by using the command with the start option -- i.e., systemctl start libvirtd. If there are still errors, check the log as was detailed previously. It can be very useful to know which version you are running, which you can find using the following command:
yum info qemu-kvm
Check VM status
Assuming the KVM service is running, you can check the VM status using the VM management tool virsh, as shown in Figure B below.
Within the virsh tool, use the list command. If there are issues connecting to the VM from outside the host -- i.e., Secure Shell -- you can get a console into the server using the virsh command console followed by the VM name. However, you have to configure a virtual serial console in the OS to use this setup. Refer to the OS documentation for how to do this.
Check disk space
When diving into KVM troubleshooting, don't underestimate the value of checking basic items, such as the amount of disk space. Just because it's a virtual environment doesn't mean troubleshooting basics don't apply.
Extracting CPU performance data is also quite simple using the following command:
virsh cpu-stats centos7
This will show the CPU performance of the VMs. Depending on the utilization values, you might need to add additional virtual CPUs. Beyond that, there is the virsh domstats <machine name> command, which will give you more detailed information about the machine.
Check memory
One of the biggest telltale signs of memory problems is the swap value. If the swap value is anything other than zero, it means that the machine is swapping out memory to physical disks. This is as bad as it gets in terms of slowing down performance.
Manually managing VMs from the command line is quite easy. Using the virsh shell, it's possible to shut down a VM using the following command:
virsh shutdown <machine name>
Similarly, to start a machine, replace the shutdown item as shown below:
virsh start <machine name>
Other commands include suspend, resume and reboot. In addition, you can tell the hypervisor to automatically start a VM by using the following command:
Virsh autostart <machine name>
Check networking
Although it's not specifically a KVM troubleshooting issue, the VM network uses Network Address Translation by default. However, in most production environments, this isn't the ideal configuration. Virtual servers usually need to be on the same network segment as the host to avoid the complexities of virtual LANs.
Be cautious of modifying the system to use bridging rather than natting because it will disrupt all the running guests. They will no longer be connected to the network and you'll need to reconfigure each and every VM. This is the kind of work that should be done ahead of time when the server is originally set up.
The configuration process can be found on the libvirt site. Make sure you understand exactly what you're doing and the effect it will have. Test the changes before trying them in production.
Use a GUI instead of virsh
If you prefer GUIs to command lines, there is a rudimentary GUI application you can use to view the console, create machines and so on. The tool in question is Virtual Machine Manager (VMM), and it can be used with any Linux distribution with a GUI installed. It enables users to do basic administration and performance measurements.
To install VMM on an Ubuntu desktop, first search for Virtual Machine Manager and then install it. You can then connect to all your KVM hosts -- use File > Connect to do this -- to get the console, see its utilization and modify its settings, as well as do rudimentary management tasks.
KVM troubleshooting isn't that much more difficult than fine-tuning any other system, but there is no substitute to having some experience with KVM and understanding how it works beyond point and click.
