Pinning VM CPUs in Proxmox

Proxmox 4 doesn't seem to support pinning a VM's CPUs to specific host CPUs. It also doesn't support VM startup hooks, so there's no straightforward way to run taskset on the newly created VM. However, when the QEMU process is created, it writes its PID to the file /var/run/qemu-server/$id.pid, where $id is the VM ID. By watching writes to this file, e.g. through inotifywait, it's actually pretty easy to create a startup hook to run taskset or perform any other tasks.

I wrote a systemd service that watches the /var/run/qemu-server directory, and automatically calls taskset on newly created VMs processes based on configuration files.

/etc/systemd/system/autotaskset.service
[Unit]
Description = Auto taskset service
 
[Service]
Type = simple
ExecStart = /bin/bash -c " \
        conf=/etc/autotaskset; \
        dir=/var/run/qemu-server; \
        mkdir -p \"$$dir\"; \
        /usr/bin/inotifywait -mq -e modify --format %%f \"$$dir\" | \
        while read pid; do \
                [ -f \"$$conf\"/\"$$pid\" ] && \
                /usr/bin/taskset $$(< \"$$conf\"/\"$$pid\") $$(< \"$$dir\"/\"$$pid\"); \
        done"
 
[Install]
WantedBy = multi-user.target

To enable the service, install the inotify-tools package, then run systemctl enable autotaskset.

To configure CPU pinning for each VM, create a file /etc/autotaskset/$id.pid, where $id is the VM ID, that contains all arguments to taskset. For example, for a VM with ID 100, if I want to pin the VM's 4 CPUs on the host's every other CPU (to skip the second hyper-threaded CPU on each physical core for example), I would use the following conf file.

/etc/autotaskset/100.pid
-cp 0,2,4,6

The -c option enables specifying the host CPU by its ID, and the -p option is required because we're operating on an existing process given its PID.

Comments

rinseaid
· 2017/04/02 22:00 · reply
Warning: "continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? in /home/public/lib/plugins/blogtng/syntax/blog.php on line 231

Warning: "continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? in /home/public/lib/plugins/blogtng/syntax/topic.php on line 89

This is awesome. Thank you very much!

Chris
· 2017/12/15 07:58 · reply

Cool!

James
· 2018/02/09 08:29 · reply

Thank you for sharing