Add lxd charm howto (#395)

* add lxd charm howto

* tweak

* Update install-lxd.md

* Confirmed the LXD installation guide

---------

Co-authored-by: Adam Dyess <adam.dyess@canonical.com>

docs/src/charm/howto/charm.md

@@ -12,6 +12,8 @@ This guide assumes the following:
 - If you still need to do this, please take a look at the quickstart
   instructions, or, for custom clouds (OpenStack, MAAS), please consult the
   [Juju documentation][juju].
+- You are not using the Juju 'localhost' cloud (see [localhost
+  instructions][localhost] for this).
 
 ```{note}
 If you cannot meet these requirements, please see the [Installing][] page for
@@ -93,3 +95,4 @@ Use `juju status` to watch these units approach the active/idle state.
 [credentials]:   https://juju.is/docs/juju/credentials
 [juju]:          https://juju.is/docs/juju/install-juju
 [charm]:         https://juju.is/docs/juju/charmed-operator
+[localhost]:     ../howto/install-lxd

docs/src/charm/howto/install-lxd.md

@@ -0,0 +1,127 @@
+# Installing to localhost/LXD
+
+The main [install instructions][install] work for most circumstances when you
+want to install Canonical Kubernetes from a charm. There are a couple of
+scenarios which require some extra steps however. These are:
+
+- deploying to the 'localhost' cloud
+- deploying to a container on a machine (i.e. when installing a bundle or using
+  the 'to:' directive to install to an existing machine)
+
+The container running the charm, or more accurately, the LXD instance
+controlling the container, needs to have a particular configuration in order
+for the Kubernetes components to operate properly.
+
+
+## Apply the Canonical Kubernetes LXD profile
+
+On the machine running the 'localhost' cloud, we can determine the existing
+profiles by running the command:
+
+```
+lxc profile list
+```
+
+For example, suppose we have created a model called 'myk8s'. This will
+output a table like this:
+
+```
++-----------------+---------------------+---------+
+|      NAME       |     DESCRIPTION     | USED BY |
++-----------------+---------------------+---------+
+| default         | Default LXD profile | 2       |
++-----------------+---------------------+---------+
+| juju-controller |                     | 1       |
++-----------------+---------------------+---------+
+| juju-myk8s      |                     | 0       |
++-----------------+---------------------+---------+
+```
+
+Each model created by Juju will generate a new profile for LXD. We can inspect
+and edit the profiles easily by using `lxc` commands.
+
+## Fetching the profile
+
+A working LXD profile is kept in the source repository for the Canonical
+Kubernetes 'k8s' snap. You can retrieve this profile by running the command:
+
+<!-- markdownlint-disable -->
+```
+wget https://raw.githubusercontent.com/canonical/k8s-snap/main/tests/integration/lxd-profile.yaml -O k8s.profile
+```
+<!-- markdownlint-restore -->
+
+To pipe the content of the file into the k8s LXD profile, run:
+
+```
+cat k8s.profile | lxc profile edit juju-myk8s
+```
+
+Remove the copied content from your directory:
+
+```
+rm k8s.profile
+```
+
+The profile editor will syntax-check the profile as part of the editing
+process, but you can confirm the contents have changed by running:
+
+```
+lxc profile show juju-myk8s
+```
+
+```{note} For an explanation of the settings in this file, [see below](explain-rules)
+```
+
+## Deploying to a container
+
+We can now deploy Canonical Kubernetes into the lxd based model as described in
+the [charm][] guide.
+
+(explain-rules)=
+
+## Explanation of custom LXD rules
+
+**boot.autostart: “true”**: Always start the container when LXD starts. This is
+needed to start the container when the host boots.
+
+**linux.kernel_modules**: Comma separated list of kernel modules to load before
+starting the container
+
+**lxc.apparmor.profile=unconfined**: Disable [AppArmor]. Allow the container to
+talk to a bunch of subsystems of the host (e.g. `/sys`). By default AppArmor
+will block nested hosting of containers, however Kubernetes needs to host
+Containers. Containers need to be confined based on their profiles thus we rely
+on confining them and not the hosts. If you can account for the needs of the
+Containers you could tighten the AppArmor profile instead of disabling it
+completely, as suggested in S.Graber's notes[^1].
+
+**lxc.cap.drop=**: Do not drop any capabilities [^2]. For justification see
+above.
+
+**lxc.mount.auto=proc:rw sys:rw**: Mount proc and sys rw. For privileged
+containers, lxc over-mounts part of /proc as read-only to avoid damage to the
+host. Kubernetes will complain with messages like `Failed to start
+ContainerManager open /proc/sys/kernel/panic: permission denied`
+
+**lxc.cgroup.devices.allow=a**: "a" stands for "all." This means that the
+container is allowed to access all devices. It's a wildcard character
+indicating permission for all devices. For justification see above.
+
+**security.nesting: “true”**: Support running LXD (nested) inside the
+container.
+
+**security.privileged: “true”**: Runs the container in privileged mode, not
+using kernel namespaces [^3], [^4]. This is needed because hosted Containers may
+need to access for example storage devices (See comment in [^5]).
+
+<!-- LINKS -->
+<!-- markdownlint-disable MD034 -->
+[^1]: https://stgraber.org/2012/05/04/
+[^2]: https://stgraber.org/2014/01/01/lxc-1-0-security-features/
+[^3]: https://unix.stackexchange.com/questions/177030/what-is-an-unprivileged-lxc-container/177031#177031
+[^4]: http://blog.benoitblanchon.fr/lxc-unprivileged-container/
+[^5]: https://wiki.ubuntu.com/LxcSecurity
+
+[AppArmor]: https://apparmor.net/
+[charm]:    ./charm