Comparing LXC vs KVM

In this blog post I want to explain and deep dive into the difference between the following two technologies: LXC and KVM. They share some similarities but are two totally different technologies and implementations which are often not quite understood in detail and what their implementation means for security, privacy and performance.

What is LXC?

LXC was initially developed at IBM and hat it's first release in 2008, nearly 15 years ago. Since then, a lot has changed, but we are only interested in the current state of LXC.

LXC describes itself like this:

LXC is a userspace interface for the Linux kernel containment features. Through a powerful API and simple tools, it lets Linux users easily create and manage system or application containers.

In practice it uses features that are available in the Linux kernel to create an isolated environment processes like cgroups and namespaces. This removes the need for a full virtualization.

LXC is regarded as an OS-Level Virtualization, meaning it's using the operating system it's running on. This has some implications.

Security

Today, LXC is considered to be secure. Every feature of the Linux kernel LXC uses is in the mainstream kernel and will be supported for a long time in the future. But there are still things to be aware of, which might create some security concerns.

If someone was to find a security issue in the Linux chroot, namespace or cgroup code an escape from the container onto the host system might be possible, allowing a user inside of the Container to execute arbitrary code on the host system. I think I don't need to explain why this is bad.

You will eventually run into limits and it seems tempting to just set the privileged option for that container, but this will start the container as the root user on the host. Meaning access to /proc and /sys is possible, so this is not safe if you do not control the container yourself.

Since all processes run directly on the host system, whoever operates the host can see all the processes that run inside of any container that's running on the host. This is not necessarily a bad thing, but can easily expose private data to individuals who are not supposed to have access to this data. Only because you control the root inside of a LXC container doesn't mean there's nothing stopping the host system administrator from snooping around in your processes and even enter your container without authentication.

Limitations

Since LXC runs on top of the existing operating system, there are things that cannot be done or cannot be abstracted.

The first major point here is operating support. You can only run LXC containers with the Linux kernel of the operating system. Meaning you won't be able to run something like Windows or FreeBSD containers in LXC on a Debian host, since Windows does not use the Linux kernel.

Also the Linux kernel cannot be changed without changing the kernel in the container, so operating system updates inside LXC cannot update the kernel and updates on the host systems kernel will change the kernel in all containers, potentially leaving unwanted side effects.

Another big problem with LXC is nesting. In the default configuration you cannot use LXC inside of LXC nor Docker or any other virtualization technology. While it is possible to allow nesting, it's not part of the default configuration and is advised against by the maintainers.

Advantages

Since there has been a lot of negativity around in the first two parts, let's checkout some advantages of using LXC over KVM.

Performance:

LXC is incredibly light-weight. Since the processes run directly on the host without any kind of abstraction, they get nearly the same performance as processes running directly on the host system. Due to some checks that take place performance is slightly less, but not noticeable. This also means that start/stop and reboot times are much lower than any full virtualization technology can be.

Lightweight:

Since LXC doesn't come with it's own kernel, the pure size of a container is much less than a fully virtualized technology can be, so it won't take up much space on your drive

Use-Case

The use-cases for LXC range far, but in my opinion it's best used if you have a containerized workload that does not rely on a specialized kernel and you control the full container. It's incredibly easy to set privileges and thus creating insecure containers but if you are the only one controlling the insides you are probably fine.

What is KVM?

KVM was developed at the Israeli company Qumranet in 2006 and was integrated into the Linux kernel in version 2.6.20 in 2007, bringing support for full virtualization on x86 platforms.

KVM is a kernel module which enables Linux to act as a hypervisor, organizing communication between a higher-level implementation like QEMU and the virtualization module of the processor (Intel VT, AMD-V).

Security

Generally, if there is not some serious bug in the processors firmware KVM is considered to be the most secure virtualization technology out there. Most cloud providers (exect Azure, they use a modified version of Hyper-V) rely on KVM to provider virtual machines to their customers. Due to the pure nature and industry-wide adoption of KVM, changes are intensely tested and even if a vulnerability is accidentally released it's usually patched very quickly.

Another benefit of a KVM is that there is no direct conneciton between the host system and the VM, meaning entering the VM and thus accessing the contents is not done easily, so customer-data inside a VM is not directly touchable even if the host system is compromised.

Limitations

Since KVM cannot directly access the kernel and it's functions, but needs to talk with the kernel to communicate with the CPU, Memory and other components of the host, performance is slightly worse than running a container. This is to be expected and in most cases with modern hardware this does not have an impact on real-world performance.

Also KVM uses more resources, since there is overhead managing the VMs so you can expect a higher base-load for an idling VM compared to an idling container.

Advantages

Possibly the biggest advantage of KVM compared to other virtualization technologies is the complete isolation of resources and thus the provided security for both the host system provider and the VM user.

The full isolation brings the benefit that you can install your own kernel in the VM. This means installing from an ISO is possible and the all of the major operating system are supported out of the Box (Linux, BSD, Windows, Mac).

The industry-wide adoption of KVM is also a great benefit due to the rigorous testing and the long-lasting maintenance it receives. Also the community around KVM and QEMU is big, so you most likely will find an answer to questions if you search.

Conclusion

In 2024, KVM is regarded as the best solution to provide virtual machines to customers. The less performance you get out of KVM is negligible on modern day hardware and the boost in security for both the host system provider and the customer outweigh other negatives by far.

If you provide hosting services to customers you definitely want to stick with KVM due to the above mentioned features.

If you are a hosting customer, you also don't want to rely on LXC. Providers can enter your any restrictions and see the processes you are running.

Proxmox

On Proxmox, you can create both LXC containers and QEMU VMs. They do give you the flexibility to decide what you want to use, which is purely depended on your use case.

If you are selling hosting services, checkout my product Proxmox Control Panel - Bennet Gallein IT Solutions . I also offer a ProxmoxCP LXC Module - Bennet Gallein IT Solutions for use in a home-lab or as an internal development platform.

Sources: