1 files changed, 148 insertions, 0 deletions
diff --git a/doc/manual.cli b/doc/manual.cli
index e20c151..e5f9b55 100644
--- a/doc/manual.cli
+++ b/doc/manual.cli
@@ -348,4 +348,152 @@ build2-mingw-x86_64-windows.tar.xz  -> build2-mingw-0.4.0-x86_64-windows.tar.xz
 While the monitor itself only needs the \c{build2-toolchain} package, build
 machine toolchain bootstrap may require additional packages (which will be
 accessed via TFTP using predictable names).
+
+\h1#machines|Build Machines|
+
+At the top level, a machine storage volume (\l{#config-storage-machines
+Machines}) contains machine directories, for example:
+
+\
+/build/machines/default/
+├── linux-gcc_6/
+└── windows-msvc_14/
+\
+
+The layout inside a machine directory is as follows, where \c{<name>} is
+the machine name and \c{toolchain} is the toolchain name:
+
+\
+<name>/
+├── <name>   -> <name>-2
+├── <name>-1/
+├── <name>-2/
+├── <name>-<toolchain>/
+└── <name>-<toolchain>-<xxx>/
+\
+
+The \c{<name>-<N>} entries are read-only \c{btrfs} subvolumes that contain
+the initial (that is, \i{pre-bootstrap}) machine images. The numeric \c{<N>}
+part indicate the machine revision.
+
+The \c{<name>} entry is a symbolic link to the \c{<name>-<N>} that is
+currently in effect.
+
+The \c{<name>-<toolchain>} entry is the bootstrapped machine image for
+\c{<toolchain>}. It is created by cloning \c{<name>} and then bootstrapping
+the \c{build2} toolchain inside.
+
+The \c{<name>-<toolchain>-<xxx>} entries are the temporary snapshots of
+\c{<name>-<toolchain>} created for building packages.
+
+A machine can be added, upgraded, or deleted on a live Build OS instance.
+This needs to be done in a particular order to avoid inconsistencies and race
+conditions.
+
+\h#machines-add|Adding a Machine|
+
+Let's assume you have a read-only \c{btrfs} \c{linux-gcc_6-1} subvolume on a
+development host (we will call it \c{devel}) that contains the initial version
+of our virtual machine. We would like to add it to the build host (running
+Build OS, we will call it \c{build}) into the default machine volume
+(\c{/build/machines/default/}). To achieve this in an atomic way we perform
+the following steps:
+
+\
+# Create the machine directory.
+#
+build$ mkdir /build/machines/default/linux-gcc_6
+
+# Send the machine subvolume to build host.
+#
+devel$ sudo btrfs send linux-gcc_6-1 | \
+  ssh build@build sudo btrfs receive /build/machines/default/linux-gcc_6/
+
+build$ cd /build/machines/default/linux-gcc_6
+
+# Make user build the owner of the machine subvolume.
+#
+build$ sudo btrfs property set -ts linux-gcc_6-1 ro false
+build$ sudo chown build:build linux-gcc_6-1
+build$ btrfs property set -ts linux-gcc_6-1 ro true
+
+# Make the subvolume the current machine.
+#
+build$ ln -s linux-gcc_6-1 linux-gcc_6
+\
+
+\h#machines-upgade|Upgrading a Machine|
+
+Continuing with the example started in the previous section, let's assume we
+have created \c{linux-gcc_6-2} as a snapshot of \c{linux-gcc_6-1} and have
+made some modification to the virtual machine (all on the development
+host). We now would like to switch to this new revision of our machine on the
+build host. To achieve this in an atomic way we perform the following steps:
+
+\
+# Send the new machine subvolume to build host incrementally.
+#
+devel$ sudo btrfs send -p linux-gcc_6-1 linux-gcc_6-2 | \
+  ssh build@build sudo btrfs receive /build/machines/default/linux-gcc_6/
+
+build$ cd /build/machines/default/linux-gcc_6
+
+# Make user build the owner of the new machine subvolume.
+#
+build$ sudo btrfs property set -ts linux-gcc_6-2 ro false
+build$ sudo chown build:build linux-gcc_6-2
+build$ btrfs property set -ts linux-gcc_6-2 ro true
+
+# Switch the current machine atomically.
+#
+build$ ln -s linux-gcc_6-2 new-linux-gcc_6
+build$ mv -T new-linux-gcc_6 linux-gcc_6
+
+# Remove the old machine subvolume (optional).
+#
+build$ btrfs property set -ts linux-gcc_6-1 ro false
+build$ btrfs subvolume delete linux-gcc_6-1
+\
+
+\h#machines-delete|Deleting a Machine|
+
+Continuing with the example started in the previous section, let's assume we
+are no longer interested in the \c{linux-gcc_6} machine and would like to
+delete it. This operation is complicated by the possibility of \c{bbot}
+instances currently building with this machine.
+
+\
+build$ cd /build/machines/default/linux-gcc_6
+
+# Delete the current machine symlink.
+#
+build$ rm linux-gcc_6
+
+# Wait for all the linux-gcc_6-<toolchain>-<xxx> subvolumes
+# to disappear.
+#
+build$ for d in linux-gcc_6-*-*; do \
+  while [ -d $d ]; do \
+    echo \"waiting for $d\" && \
+    sleep 10; \
+  done; \
+done
+
+# Delete the initial and bootstrapped machine subvolume(s).
+#
+build$ for d in linux-gcc_6-*; do \
+  btrfs property set -ts $d ro false && \
+  btrfs subvolume delete $d; \
+done
+
+# Delete the machine directory.
+#
+build$ cd ..
+build$ rmdir /build/machines/default/linux-gcc_6
+\
+
+Note also that on reboot the Build OS monitor examines and cleans up
+machine directories of any stray subvolumes. As a result, an alternative
+approach would be to remove the current machine symlink and reboot the
+build host.
 "