Add policykit rules for virt-manager

If a user wants to use virt-man to control virtual machines on localhost, he will be prompted to authenticate to policykit.

If you have a policykit rule established (as the libvirt package already provides) for the right group, it’s seamless. Just make sure the user is in group “libvirt” (from /usr/share/polkit-1/rules.d/50-libvirt.rules)
Or, to make a different group the (additional) accepted value:

tf=/etc/polkit-1/rules.d/80-libvirt.rules
touch "${tf}" ; chmod 0644 "${tf}"
cat <<EOF >"${tf}"
polkit.addRule(function(action, subject) {
  if (action.id == "org.libvirt.unix.manage" && subject.local && subject.active && subject.isInGroup("wheel")) {
      return polkit.Result.YES;
  }
});
EOF
Advertisements

Sysprep for Linux

Named after a famous non-free OS sysprep process, my GNU/Linux sysprep process helps my company when I need to clone existing vms. For some reason my apps teams are incapable of installing their applications again, so they just have me clone the system. I blank it out as much as possible, and then provision it like it’s new.

Deploy the sysprep.sh script to the original server and add a cronjob.

    th=originalserver350
    scp -p /etc/ansible/books/test/sysprep/files/sysprep.sh ${th}:~/
    ssh ${th}
    sudo su -
    /bin/cp -pf /home/*/sysprep.sh /root/sysprep.sh ; chmod 0755 /root/sysprep.sh ; chown root.root /root/sysprep.sh
    tf=/etc/cron.d/00_sysprep.cron
    touch "${tf}" ; chown root.root "${tf}" ; chmod 0600 "${tf}"
    cat <<-EOFCRON 1>"${tf}"
@reboot         root    test ! "\$( hostname -s )" = "$( hostname -s )" && test -e /root/sysprep.sh && sleep 20 && /bin/sh -x /root/sysprep.sh cron 1> /root/intermediate-clone.log 2>&1
EOFCRON
    exit # leave root shell
    exit # leave ssh to original server

Inject the regular server deployment scripts.
Modify the td and tv variables here as needed.

    td=/etc/ansible/books/stable/clone
    tv=RHEL7
    pushd "${td}"
    tar -zc -C /etc/ansible/roles/localization -f /tmp/localizations.tgz files
    scp -p /tmp/localizations.tgz "${th}":~
    popd
    pushd "${td}/files/${tv}"
    scp -p *.sh "${th}":~
    popd
    ssh "${th}"
    sudo su -
    /bin/mv -f /home/bgirton/*.sh /root/ ; chown root.root /root/*.sh ; chmod 0750 /root/*.sh ;
    tar -zx -C /etc -f /home/*/localizations.tgz ;
    test -e /etc/files && ! test -e /etc/templates && /bin/mv -f /etc/files /etc/templates ;
    chown -R root:root /etc/templates 2>/dev/null ;
    exit # leave root shell
    exit # leave ssh to original server

The cronjob to run the s1_init script will be provided by the intermediate template’s execution of the sysprep script.

Upon cloning the vm, the sysprep script will be executed! Here it is:

#!/bin/sh
# File: sysprep.sh
# Location: /etc/ansible/books/test/sysprep/files/
# Author: bgstack15
# Startdate: 2018-10-11 11:39
# Title: Script that Syspreps an Intermediate Server
# Purpose: Convert an intermediate clone of an existing machine to make it more suitable for use as a VM template
# History:
# Usage:
#    On the intermediate template, run this shell script and then power off the vm.
# Reference:
#    https://www.cyberciti.biz/tips/vmware-linux-lost-eth0-after-cloning-image.html
#    https://unix.stackexchange.com/questions/39370/how-to-reload-udev-rules-without-reboot
# Improve:
# Documentation:

echo "sysprep.sh version 2018-10-12b"

# make verbose
set -x

export PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

# continue only if hostname indicates this is an intermediate template
hostname -s | grep -qiE 'tmp$' || exit 0

# clean up cronjob so this script is only called that very first boot of the intermediate template
/bin/rm -f /etc/cron.d/00_sysprep.cron

# prepare cronjob for s1_init for a normal clone deployment
# this is mostly for the eth0 renaming again
tf=/etc/cron.d/01_init.cron
touch "${tf}" ; chown root.root "${tf}" ; chmod 0600 "${tf}"
cat <<-EOFCRON 1>"${tf}"
@reboot         root    test ! "\$( hostname -s )" = "$( hostname -s )" && test -e /root/s1_init.sh && /bin/sh -x /root/s1_init.sh cron 1> /root/clone.log 2>&1
EOFCRON

# fix rootsh
this_rootsh="$( which rootsh 2>/dev/null )"
tf=/usr/local/bin/root
touch "${tf}" ; chmod 0755 "${tf}" ; chown root.root "${tf}"
cat < "${tf}"
if test "\$USER" = "root" ; then
  shift
  \$@
else
  /usr/bin/sudo ${this_rootsh:-/usr/local/bin/rootsh} \$@ --no-syslog
fi
EOF

# disable domain connectivity
/bin/rm -f /etc/krb5.keytab /etc/krb5.conf*
service sssd stop ; systemctl stop sssd.service ;
chkconfig sssd off ; systemctl disable sssd.service ;

# disable monitoring
for ts in ccmexecd scx-cimd ;
do
   service "${ts}" stop ; systemctl stop "${ts}.service"
   chkconfig "${ts}" off ; systemctl disable "${ts}.service"
done
# remove old ssl certificates
rm -f /etc/opt/microsoft/scx/ssl/*

# clean old log and config files
pushd /var/log
rm -f boot.log* btmp* cron* dmesg* dracut* maillog* messages* prepatch/* rootsh/* sa/* samba/* secure* spooler/* sssd/* syslog* vmware-* wtmp*
popd
pushd /root
rm -f core* *.ansible_backup KASetup*log log.*.txt VMwareTools*gz
rm -rf vmware-tools-distrib
popd
pushd /etc
rm -f *.cfsaved fstab.* fstab-* hosts.20*
popd
# clean ansible bupped files
rm -f *.20*@*~
# clean homedirs and bash histories
find /home -maxdepth 1 -mindepth 1 ! -name 'ansible' ! -name 'bgstack15' -exec rm -rf {} \;
for tf in /home/*/.bash_history ;
do
   cat /dev/null | tee "${tf}"
done

# fix networking, rhel6 on vmware
# from project clone RHEL6/s1_init.sh
nic_count="$( ip -o l | grep -cE 'eth[0-9]' )"
this_nic="$( ip -o l | grep -oE 'eth[0-9]' | sort | tail -n1 )"
out_nic=eth0
out_nic_file="/etc/sysconfig/network-scripts/ifcfg-${out_nic}"
this_nic_file="/etc/sysconfig/network-scripts/ifcfg-${this_nic}"
test "${nic_count}" -eq 1 && test "${this_nic}" != "${out_nic}" && {
   sed -i -r -e "s/${this_nic}/${out_nic}/g;" "${this_nic_file}" 2>/dev/null
   sed -i -r -e "s/${this_nic}/${out_nic}/g;" "${out_nic_file}" 2>/dev/null
   test -e "${this_nic_file}" && /bin/mv -f "${this_nic_file}" "${out_nic_file}"
}
/bin/rm -f /etc/udev/rules.d/70-persistent-net.rules
udevadm control --reload-rules && udevadm trigger
# the change will take effect upon next reboot

# fix iptables rule tied to eth0
sed -i -r -e '/dport 1270/s/-i eth[0-9]* //;' /etc/sysconfig/iptables

# generate new ssh host keys
# but this will also happen in the s1_init script
/bin/rm -rf /etc/ssh/*key*
# but do not actually generate them, because this is still just the intermediate template
#/sbin/service sshd stop ; /bin/sleep 5 ; /sbin/service sshd start ;

# clean up old sudoers backups
rm -f /etc/sudoers.20* /etc/sudoers.cf* /etc/sudoers.rpm* /etc/sudoers.d/disable_requiretty.*

# when this process is verified, the template should shut itself down
shutdown -h now ; systemctl poweroff ;

Instructions for Setting Up a CentOS 7 System with Bridged Networking for Virtual Machines

CentOS 7 bridging network card for virtual machines

My goal is to set up virtualization where the guests can access the entire LAN as well as the host over the network. The host should also be able to reach all the guests via the network.

This task was so simple, but somehow it eluded me for over a year. I use this document to establish a new kvm host in my network pool.

Install virtualization tools

sudo yum -y install libvirt qemu-kvm virt-install
sudo systemctl enable libvirtd.service ; sudo systemctl start libvirtd.service
sudo setsebool -P virt_use_nfs 1

Adjust the ethernet configuration

sudo su -
{
this_nic="$( nmcli device show | awk '/^GENERAL.DEVICE:/ && $2 ~ /e.*/ {print $2}' )"
indir=/etc/sysconfig/network-scripts
this_bridge=br0
this_nic_count="$( printf "%s\n" "${this_nic}" | sed '/^\s*$/d' | wc -l )"
if test ${this_nic_count} -ne 1 ;
then
   echo "Other than 1 nic detected. Please deal with manually. Aborted."
else
   # prepare values for bridge definition
   this_mac="$( ip -o link | grep "${this_nic}" | grep -oE 'ether [a-fA-F0-9:]{17}' | awk '{print $2}' | tr '[[:lower:]]' '[[:upper:]]' )"
   this_ipaddr="$( ip -o address show "${this_nic}" | grep -oE 'inet [0-9\.]{7,15}' | awk '{print $2}' )"
   # define bridge interface
   {
      echo "DEVICE=${this_bridge}"
      echo "TYPE=Bridge"
      echo "ONBOOT=yes"
      echo "DELAY=0"
      grep -h -E 'DNS1|DNS2|DOMAIN|IPADDR|PREFIX|BOOTPROTO|GATEWAY|DEFROUTE' "${indir}/ifcfg-${this_nic}"
   } > "${indir}/ifcfg-${this_bridge}"
   # define ethernet card
   old_nic_file="${indir}/ifcfg-${this_nic}"
   temp_nic_file="${indir}/ifcfg-${this_nic}-new"
   {
      echo "DEVICE=${this_nic}"
      echo "HWADDR=${this_mac}"
      echo "ONBOOT=yes"
      echo "BRIDGE=${this_bridge}"
      grep -h -E 'UUID' "${old_nic_file}"
   } > "${temp_nic_file}"
   chmod --reference "${old_nic_file}" "${temp_nic_file}"
   /bin/mv -f "${temp_nic_file}" "${old_nic_file}"
fi
}

systemctl restart network.service NetworkManager.service

Using the virtual host

With the setup complete, the environment is ready to serve virtual machines!

Install a virtual machine

vm=c7-03a ; time sudo virt-install -n "${vm}" --memory 2048 --vcpus=1 --os-variant=centos7.0 --accelerate -v --disk path=/var/lib/libvirt/images/"${vm}".qcow2,size=20 -l /mnt/public/Support/SetupsBig/Linux/CentOS-7-x86_64-Minimal-1804.iso --initrd-inject=/mnt/public/Support/Platforms/CentOS7/centos7-ks.cfg --extra-args "ks=file:/centos7-ks.cfg SERVERNAME=${vm} NOTIFYEMAIL=bgstack15@ipa.example.com" --debug --network type=bridge,source=br0 --noautoconsole

Delete a virtual machine

vm=c7-03a; sudo virsh destroy "${vm}"; sudo virsh undefine --remove-all-storage "${vm}";

References

Weblinks

  1. https://wiki.libvirt.org/page/Networking

Internal files

  1. file:///mnt/public/Support/Platforms/CentOS7/centos7-ks.cfg

Multiple monitors on Windows guest in KVM

Introduction

It is easy to set up a virtual machine with the virt-manager GUI.

To add a second monitor (or more) is also pretty easy, once you know how to do it. However, to view a second monitor simultaneously with the first, you will need to use the tool remote-viewer.

In virt-manager, select “Show virtual hardware details.”
Screenshot of virt-manager open to a virtual machine, "Show virtual hardware details" page.
Add a new video card. A basic QXL type should be sufficient.

It is possible to connect to the guest’s displays over the network, if you configure it to be possible. For example, you use your desktop Virtual Machine Manager to connect to a server’s libvirt via a connection string like qemu+ssh://root@vm1.ipa.example.com/system.

On that virtual machine’s “Display spice” virtual hardware, modify the address tag to “All interfaces.” Also note the the port number given to this guest. In my screenshot you can see mine is port 5907.
Screenshot showing vm settings for display spice

You will want to open up the firewall on the vm host. I suggest just using the vdsm definition, which is for the oVirt project and includes TCP ports 5900-6923.

sudo firewall-cmd --permanent --add-service=vdsm; sudo firewall-cmd --reload

You will need to shut down (not reboot) the guest if it is running at the time, for it to be able to use the new virtual hardware or pretty much any new setting.

Once the virtual machine is running again, use “Remote Viewer” in the GUI, or run from the command line.

remote-viewer spice://vm1.ipa.example.com:5907

Improvements

Learn how to do this in the cli, including maybe at the virt-install statement. Or at least how to retro-fit an existing domain.

References

  1. search “kvm spice guest windows multiple monitors”
  2. Shamelessly ripped off from https://solus-project.com/forums/viewtopic.php?f=11&t=8663
  3. spice guest tools https://www.spice-space.org/download.html

Create, attach, detach disk to vm in kvm on command line

In kvm, let’s say you want to create a new disk and attach it to a virtual machine. You could use Virtual Machine Manager. But for the command line, here is a quick way how to do it.

Create a disk

Create a qcow2 disk that is fully allocated. When I tried with disks that were not fully allocated, the vm would only see the 200K or so and would not let me write a partition table large enough to do anything.

time qemu-img create -f qcow2 /var/lib/libvirt/images/vmname-vdb.qcow2 22000M -o preallocation=full

Attach a disk

You can omit the flags as needed, if you don’t want to update the virtual machine’s definition, or –config. And some of these are probably redundant, but they did not throw errors for me and I wanted the change to be immediate and persistent upon vm reboot, so I leave them all in.

virsh attach-disk --domain vmname /var/lib/libvirt/images/vmname-vdb.qcow2 --target vdb --persistent --config --live

Detach a disk

Same thing as the attaching, only you don’t include the –target flag.

virsh detach-disk --domain vmname /var/lib/libvirt/images/vmname-vdb.qcow2 --persistent --config --live

References

Weblinks

Man pages

  1. qemu-img

Remove scsi disk from operating system

To tell an OS you are ready to remove a scsi disk, you can run this command:

echo 1 > /sys/block/sdX/device/delete

Where sdX is the device you are ready to detach.

For virtual machines I don’t usually bother doing that, but that is how to safely tell GNU/Linux you are going to remove the device.

References

Weblinks

  1. Original question https://unix.stackexchange.com/questions/31029/echo-1-sys-block-sdx-device-delete-on-all-disks-except-predetermined-list

X forwarding for virt-manager to Windows

Story time! When I was working on my virtual environment, I rebooted my main desktop. So I was stuck using my Windows desktop for a minute, and I wanted to work on my virtual machines.

I decided to do some X forwarding, which for virt-manager on CentOS 7 needs some special steps.

On the server

The first thing is to install virt-manager. You also will need a piece of software named xauth, and some special fonts.

yum install -y virt-manager xauth dejavu-\*fonts

Reference: https://superuser.com/questions/119792/how-to-use-x11-forwarding-with-putty/119908#119908

On the client

On the Windows client, you should install an X server. I picked Xming. It also needs it fonts installed.

Run Xming.

Connect to server with PuTTY. You will need to configure PuTTY to allow X forwarding, and to use the right X server.

Screenshot of Putty configuration screen showing X11 forwarding options
Telling PuTTY to allow X11 forwarding to localhost:0

Notes

Unfortunately, my keyboard input to the virtual machine does not work when I have it configured with a spice display. But it works when I use a vnc display. See https://bugzilla.redhat.com/show_bug.cgi?id=1236412 for a closed bug report that has similar symptoms to this problem.

virt-manager settings of a virtual machine, showing where to change display to VNC server type.
Setting on virtual machine of VNC server or spice server.

References

Weblinks

  1. http://straightrunning.com/XmingNotes/#head-131
  2. https://superuser.com/questions/119792/how-to-use-x11-forwarding-with-putty/119908#119908
  3. https://sourceforge.net/projects/xming/files/Xming/
  4. https://sourceforge.net/projects/xming/files/Xming-fonts/
  5. https://robert.penz.name/354/how-to-fix-the-font-for-virt-manager-via-x-forwarding/
  6. https://bugzilla.redhat.com/show_bug.cgi?id=1236412
  7. PuTTY https://www.chiark.greenend.org.uk/~sgtatham/putty/

Virsh get total cpu allocations

tl;dr

virsh list | awk '{print $1}' | grep -oIE "[0-9]*" | while read word; do virsh dominfo ${word} | grep "CPU.s"; done | awk 'BEGIN {a=0;} {a=a+$2;} END {print a;}'

The explanation

If you want to get the total allocation of vCPUs to all the guests on a kvm host, you can use this one-liner.
virsh list gets the list of running domains (virtual machines).
The awk and grep get only the domain id numbers (could do it by domain name if you wish).
virsh dominfo gets the cpu allocation for each listed domain, by iterating over the list.
The final awk statements counts the numbers.

Get total physical CPUs available

virsh nodeinfo
CPU model:           x86_64
CPU(s):              24
CPU frequency:       1899 MHz
CPU socket(s):       1
Core(s) per socket:  6
Thread(s) per core:  2
NUMA cell(s):        2
Memory size:         198310648 KiB