Overview
This blog post will focus on the following points, you can skip to the point you want, they don't require any of the others. It is recommended to at least read everything, even if you think you have a good setup - there might be some more to squeeze out.
- CPU Pinning
- Memory Management
- Storage Optimization
- Network Optimization
- Resource Allocation Strategies
CPU Pinning
CPU pinning is a technique that allows you to assign specific virtual CPUs (vCPUs) to physical CPU cores. This can significantly improve performance, especially for latency-sensitive applications by having cores dedicated to VMs that need lower latency. Benefits includes reduced context switching, improved cache utilization and lower, more predictable latency.
There are a lot of resources out there on how NUMA and CPU pinning is implemented correctly and I suggest you read up on that elsewhere. Maybe sometime in the future there will be a blog post here, but the topic is huge and would blow the context of this post. Just be aware that it exists and that it'll help you a lot in certain senarios.
Memory Managment
Effective memory management is crucial for optimal Proxmox performance. Two key techniques to focus on are huge pages and memory ballooning. To enable huge pages, edit your "/etc/default/grub" file to include "hugepages=1024" in "GRUB_CMDLINE_LINUX_DEFAULT". Use "update-grub" and reboot your server. Now you have huge tables enabled, meaning memory reading is done in bigger chunks, saving time when a bigger chunk is read.
You also can enable memory ballooning on linux VMs (usually not recommended for Windows VMs because of performance reasons), which means that memory will be dynamically allocated to VMs via the use of a special kernel module inside the VM.
Storage
Choosing the right storage for your use-case will have the biggest impact on how fast your VM can access files. Proxmox supports many different storage backends, like CEPH, ZFS, LVM and more. There are pros and cons for every different storage, but there are some key considerations you have to take. If you have a single disk, don't use something like ZFS. You get some nice features on top of it, but performance will be impacted in comparison to something more lightweight, like LVM. If you have multiple disks or perhaps even a cluster with more than 3 Nodes, you could use CEPH, but CEPH requires fast disks + blazing fast, dedicated networking. We're talking 10GbE at least on a dedicated network, just for ceph to have reasonable performance that would justify the performance gain compared to ZFS.
So when deciding on your storage layout, research the implemented methods and their pros and cons to decide what's the best for your use-case.
Network Optimization
Advanced networking configurations can significantly boost VM network performance.
SR-IOV (Single Root I/O Virtualization) allows a single physical network interface to be shared among multiple VMs. This can be enabled and will improve network latency on the VM and resource usage on the host, since networking does not pass through the host. But this also means that you cannot use the Proxmox Firewall.
Resource Allocation Strategies
Balancing resources across VMs is crucial for overall cluster performance.**CPU Shares and Limits:**
- Use CPU shares to prioritize VMs
- Set CPU limits to prevent resource hogging
Memory Overcommitment:
- Carefully overcommit memory based on workload patterns
- Use KSM (Kernel Samepage Merging) to reduce memory usage
I/O Throttling:
- Set I/O limits to prevent a single VM from saturating storage
- Use `ionice` for fine-grained I/O prioritization
# Set CPU and I/O limits qm set 100 -cpulimit 50 qm set 100 -iothread 1 qm set 100 -scsihw virtio-scsi-pci qm set 100 -virtio0 local-lvm:vm-100-disk-0,iothread=1,mbps_rd=50,mbps_wr=50
By implementing these advanced tuning techniques, Proxmox administrators can significantly enhance the performance and efficiency of their virtualization environments. Each of these topics could potentially be expanded into its own detailed blog post, providing readers with in-depth knowledge and practical implementation steps.