1---
2title: "LXD: Containers for Human Beings"
3subtitle: "Docker's great and all, but I prefer the workflow of interacting with VMs"
4date: 2023-08-11T16:30:00-04:00
5categories:
6 - Technology
7tags:
8 - Sysadmin
9 - Containers
10 - VMs
11 - Docker
12 - LXD
13draft: true
14rss_only: false
15cover: ./cover.png
16---
17
18This is a blog post version of a talk I presented at both Ubuntu Summit 2022 and
19SouthEast LinuxFest 2023. The first was not recorded, but the second was and is
20on [SELF's PeerTube instance.][selfpeertube] I apologise for the terrible audio,
21but there's unfortunately nothing I can do about that. If you're already
22intimately familiar with the core concepts of VMs or containers, I would suggest
23skipping those respective sections. If you're vaguely familiar with either, I
24would recommend reading them because I do go a little bit in-depth.
25
26[selfpeertube]: https://peertube.linuxrocks.online/w/hjiTPHVwGz4hy9n3cUL1mq?start=1m
27
28{{< adm type="warn" >}}
29
30**Note:** Canonical has decided to [pull LXD out][lxd] from under the Linux
31Containers entity and instead continue development under the Canonical brand.
32The majority of the LXD creators and developers have congregated around a fork
33called [Incus.][inc] I'll be keeping a close eye on the project and intend to
34migrate as soon as there's an installable release.
35
36[lxd]: https://linuxcontainers.org/lxd/
37[inc]: https://linuxcontainers.org/incus/
38
39{{< /adm >}}
40
41## The benefits of VMs and containers
42
43- **Isolation:** you don't want to allow an attacker to infiltrate your email
44 server through your web application; the two should be completely separate
45 from each other and VMs/containers provide strong isolation guarantees.
46- **Flexibility:** <abbr title="Virtual Machines">VMs</abbr> and containers only
47 use the resources they've been given. If you tell the VM it has 200 MBs of
48 RAM, it's going to make do with 200 MBs of RAM and the kernel's <abbr
49 title="Out Of Memory">OOM</abbr> killer is going to have a fun time 🤠
50- **Portability:** once set up and configured, VMs and containers can mostly be
51 treated as black boxes; as long as the surrounding environment of the new host
52 is similar to the previous in terms of communication (proxies, web servers,
53 etc.), they can just be picked up and dropped between various hosts as
54 necessary.
55- **Density:** applications are usually much lighter than the systems they're
56 running on, so it makes sense to run many applications on one system. VMs and
57 containers facilitate that without sacrificing security.
58- **Cleanliness:** VMs and containers are applications in black boxes. When
59 you're done with the box, you can just throw it away and most everything
60 related to the application is gone.
61
62## Virtual machines
63
64As the name suggests, Virtual Machines are all virtual; a hypervisor creates
65virtual disks for storage, virtual <abbr title="Central Processing
66Units">CPUs</abbr>, virtual <abbr title="Network Interface Cards">NICs</abbr>,
67virtual <abbr title="Random Access Memory">RAM</abbr>, etc. On top of the
68virtualised hardware, you have your kernel. This is what facilitates
69communication between the operating system and the (virtual) hardware. Above
70that is the operating system and all your applications.
71
72At this point, the stack is quite large; VMs aren't exactly lightweight, and
73this impacts how densely you can pack the host.
74
75I mentioned a "hypervisor" a minute ago. I've explained what hypervisors in
76general do, but there are actually two different kinds of hypervisor. They're
77creatively named **Type 1** and **Type 2**.
78
79### Type 1 hypervisors
80
81These run directly in the host kernel without an intermediary OS. A good example
82would be [KVM,][kvm] a **VM** hypervisor than runs in the **K**ernel. Type 1
83hypervisors can communicate directly with the host's hardware to allocate RAM,
84issue instructions to the CPU, etc.
85
86[debian]: https://debian.org
87[kvm]: https://www.linux-kvm.org
88[vb]: https://www.virtualbox.org/
89
90```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
91hk: Host kernel
92hk.h: Type 1 hypervisor
93hk.h.k1: Guest kernel
94hk.h.k2: Guest kernel
95hk.h.k3: Guest kernel
96hk.h.k1.os1: Guest OS
97hk.h.k2.os2: Guest OS
98hk.h.k3.os3: Guest OS
99hk.h.k1.os1.app1: Many apps
100hk.h.k2.os2.app2: Many apps
101hk.h.k3.os3.app3: Many apps
102```
103
104### Type 2 hypervisors
105
106These run in userspace as an application, like [VirtualBox.][vb] Type 2
107hypervisors have to first go through the operating system, adding an additional
108layer to the stack.
109
110```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
111hk: Host kernel
112hk.os: Host OS
113hk.os.h: Type 2 hypervisor
114hk.os.h.k1: Guest kernel
115hk.os.h.k2: Guest kernel
116hk.os.h.k3: Guest kernel
117hk.os.h.k1.os1: Guest OS
118hk.os.h.k2.os2: Guest OS
119hk.os.h.k3.os3: Guest OS
120hk.os.h.k1.os1.app1: Many apps
121hk.os.h.k2.os2.app2: Many apps
122hk.os.h.k3.os3.app3: Many apps
123```
124
125## Containers
126
127As most people know them right now, containers are exclusive to Linux.[^1] This is
128because they use namespaces and cgroups to achieve isolation.
129
130- **[Linux namespaces]** partition kernel resources like process IDs, hostnames,
131 user IDs, directory hierarchies, network access, etc.
132- **[Cgroups]** limit, track, and isolate the hardware resource use of a set of
133 processes
134
135[Linux namespaces]: https://en.wikipedia.org/wiki/Linux_namespaces
136[Cgroups]: https://en.wikipedia.org/wiki/Cgroups
137
138### Application containers
139
140```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
141Host kernel.Container runtime.c1: Container
142Host kernel.Container runtime.c2: Container
143Host kernel.Container runtime.c3: Container
144
145Host kernel.Container runtime.c1.One app
146Host kernel.Container runtime.c2.Few apps
147Host kernel.Container runtime.c3.Full OS.Many apps
148```
149
150### System containers
151
152```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
153hk: Host kernel
154hk.c1: Container
155hk.c2: Container
156hk.c3: Container
157hk.c1.os1: Full OS
158hk.c2.os2: Full OS
159hk.c3.os3: Full OS
160hk.c1.os1.app1: Many apps
161hk.c2.os2.app2: Many apps
162hk.c3.os3.app3: Many apps
163```
164
165## When to use VMs
166
167- Virtualising esoteric hardware
168- Virtualising non-Linux operating systems (Windows, macOS)
169- Completely isolating processes from one another with a decades-old, battle-tested technique
170
171{{< adm type="note" >}}
172See Drew DeVault's blog post [_In praise of qemu_](https://earl.run/rmBs) for a great use of VMs
173{{< /adm >}}
174
175### When you use application containers
176
177- Microservices
178- Extremely reproducible builds
179 - (NixOS.org would likely be a better fit though)
180- Dead-set on using cloud platforms with extreme scaling capabilities (AWS, GCP, etc.)
181- When the app you want to run is _only_ distributed as a Docker container and
182 the maintainers adamantly refuse to support any other deployment method
183 - (Docker does run in LXD 😉)
184
185### System containers
186
187- Anything not listed above 👍
188
189## Crash course to LXD
190
191### Installation
192
193{{< adm type="note" >}}
194
195**Note:** the instructions below say to install LXD using [Snap.][snap] I
196personally dislike Snap, but LXD is a Canonical product and they're doing their
197best to prmote it as much as possible. One of the first things the Incus project
198did was [rip out Snap support,][rsnap] so it will eventually be installable as a
199proper native package.
200
201[snap]: https://en.wikipedia.org/wiki/Snap_(software)
202[rsnap]: https://github.com/lxc/incus/compare/9579f65cd0f215ecd847e8c1cea2ebe96c56be4a...3f64077a80e028bb92b491d42037124e9734d4c7
203
204{{< /adm >}}
205
2061. Install snap following [Canonical's tutorial](https://earl.run/ZvUK)
207 - LXD is natively packaged for Arch and Alpine, but configuration can be a
208 massive headache.
2092. `sudo snap install lxd`
2103. `lxd init`
2114. `lxc image copy images:debian/11 local: --alias deb-11`
2125. `lxc launch deb-11 container-name`
2136. `lxc shell container-name`
214
215### Usage
216
217{install my URL shortener}
218
219[^1]: Docker containers on Windows and macOS actually run in a Linux VM.