Add private install README

1 files changed, 270 insertions, 0 deletions

diff --git a/etc/private/README b/etc/private/README
new file mode 100644
index 0000000..22255ae
--- /dev/null
+++ b/etc/private/README
@@ -0,0 +1,270 @@
+This directory contains a virtual machine (VM) with a build2 repository web
+interface (brep) installation for a private package repository plus a number
+of scripts and configuration files for running this machine as a systemd
+service.
+
+A brep installation consists of a web server (Apache2), a database server
+(PostgreSQL), and a number of auxiliary processes (repository loader,
+submission handler, etc). While all this can be installed and configured
+manually (as described in brep/INSTALL), this VM makes it possible to quickly
+setup a private repository.
+
+Note that the configuration offered by this VM is only suitable for a
+private/trusted environment, normally either for personal use (host-local)
+or for use within an organization's private network. Specifically:
+
+  - The repository is accessed via HTTP.
+
+  - The repository is not signed.
+
+  - Submitted packages are published directly and without ownership
+    authentication.
+
+  - The VM does not perform auto-update (since it does not assume the presence
+    of an Internet connection) and therefore may not have the latest security
+    updates.
+
+The below setup instructions are for host machines that run systemd-based
+Linux distributions. Note, however, that it should be possible to use other
+distributions or operating systems provided they are capable of running
+QEMU/KVM virtual machines. The following utilities are expected to be
+available on the host machine:
+
+  - systemd >= 229    (systemd --version)
+  - bash    >= 4.3    (base --version)
+  - qemu    >= 2.5.0  (qemu-system-x86_64 --version)
+  - screen, socat     (screen --version, socan -V)
+
+Consult your distribution's package manager if any of these utilities are
+missing.
+
+The host machine is also expected to have KVM virtualization support as
+well as at least 1G or RAM (2G recommended) and at least 5G of disk space
+(4G for VM image and the rest for package storage) that can be dedicated
+to the VM.
+
+Note: commands that start with the `#` prompt must be executed as root.
+
+
+1. Create the brep user and group.
+
+In this setup, the VM image, scripts, etc., as well as the repository packages
+are all kept in the home directory of the special brep user. In particular,
+the packages are stored on the host machine (as opposed to inside the VM
+image) and are shared with the VM (using the virtio-9p filesystem). As a
+result, if necessary, you can manipulate the package repository from the host
+machine (but see Step 6 below for the rules). This setup also makes it easier
+to upgrade VM images by simply replacing the old one with a new.
+
+However, to make this arrangement work reliably, the brep user/group IDs on
+the host machine must match those inside the VM. As a result, we create the
+brep user with specific IDs:
+
+# addgroup --gid 63700 brep
+# adduser --uid 63700 --gid 63700 --gecos '' --disabled-password brep
+
+Additionally, if your distribution requires users that are allowed to use KVM
+to belong to a special group (normally kvm), then add the brep user to this
+group:
+
+# usermod -G kvm brep
+
+If unsure whether this is required, skip this step and come back to it if you
+get 'KVM: permission denied' error on Step 4.
+
+
+2. Download and unpack the VM archive into the brep user's home directory.
+
+# su - brep
+$ pwd
+/home/brep
+
+$ curl -fO https://download.build2.org/X.Y.Z/linux-debian-N-brep-X.Y.Z.tar.xz
+$ tar xf --strip-components=1 linux-debian-N-brep-X.Y.Z.tar.xz
+$ ls
+bin/ etc/ vm/ vm-brep@.service
+
+
+3. Configure networking for the VM.
+
+This setup expects the VM to use bridged networking with a persistent tap
+interface. This allows for a wide variety of configurations ranging between
+host-local (private bridge without routing), subnet (private bridge with
+routing/NAT), and local network (public bridge over host's Ethernet
+adapter). In particular, the last configuration would make the repository
+accessible from other machines on the same local network.
+
+The exact steps on how to setup bridged networking and create a persistent tap
+interface depend the network manager used so consult your distribution's
+documentation for details. The guide found in etc/systemd-networkd/README
+shows how to setup the local network configuration mentioned above using the
+systemd-networkd network manager available on most systemd-based
+distributions.
+
+In the rest of this guide we assume that tap interface called tap0 is
+appropriately configured and is owned by user/group brep.
+
+
+4. Generate a MAC address and start the VM for testing.
+
+The recommended way to obtain a MAC address for the VM is to generate it
+based on the host's Ethernet adapter address (see inside vm-gen-macaddress
+for details):
+
+$ ~/bin/vm-gen-macaddress xx:yy:yy:yy:yy:yy 0
+
+Where xx:yy:yy:yy:yy:yy is the MAC address of the host's Ethernet adapter that
+can be viewed with the following command:
+
+# ip link show
+
+The address printed by vm-gen-macaddress will be in the 02:yy:yy:yy:yy:yy
+form.
+
+If you are using a local network configuration, then now is a good time to
+assign the VM its IP address and hostname. If you need to submit a request to
+your network administrator, then the following text could serve as a template:
+
+"I would like to run a VM on the <host> machine that needs to have its own IP
+ address and domain name (configured via DHCP. It will have fixed MAC address
+ <mac> (which was derived from <host>'s physical Ethernet address; but you are
+ welcome to assign a different MAC address if desired). I would like this
+ machine to have the <vm> name if possible.
+
+ This will be a QEMU/KVM virtual machine running as a systemd service. It will
+ use bridged networking with a tap interface."
+
+Where:
+
+  <host>  host machine name, for example, myserver.lan (run hostname -f)
+  <mac>   the generated mac address (02:yy:yy:yy:yy:yy)
+  <vm>    VM machine name, for example, mybrep.lan
+
+Note that the VM is configured to receive its hostname from DHCP. Failed that,
+the repository URL will use the IP address.
+
+Next, create the package repository directory and start the VM for testing
+(replace 02:yy:yy:yy:yy:yy with the actual MAC address):
+
+$ mkdir -p state/bpkg
+$ ~/bin/vm-start --stdio --tap tap0 --mac 02:yy:yy:yy:yy:yy vm/brep.img
+
+After booting, you will be presented with a login. Login as root with password
+123 (VM command prompts are shown indented with two spaces). Then verify IP
+address, hostname, and the network functionality:
+
+  # ip addr show
+  # hostname -f
+  # ping example.org
+
+If everything appears correct, visit the repository web page with a browser
+(for example, http://mybrep.lan). Check the About page to see the repository
+URL.
+
+Try to submit a package (for example, from your development machine):
+
+$ bdep new hello
+$ cd hello
+$ git add . && git commit -m test
+$ bdep init -C @test cc
+$ bdep publish --control=none --repository http://mybrep.lan --force=snapshot
+
+Visit the repository web page and confirm the package is there. Then try to
+build the submitted package from the repository:
+
+$ bpkg create -d test
+$ bpkg build -d test hello@http://mybrep.lan/1
+
+If everything is working fine, shut the VM down:
+
+  # shutdown -h now
+
+
+5. Setup the VM to run as a systemd service.
+
+To start the VM as a systemd service on host boot, perform the following
+steps.
+
+First, create the VM configuration file (replace 02:yy:yy:yy:yy:yy with the
+actual MAC address)::
+
+$ cat <<EOF >vm/brep.conf
+RAM=2G
+CPU=1
+TAP=tap0
+MAC=02:yy:yy:yy:yy:yy
+EOF
+
+Then configure the systemd service:
+
+# cp ~brep/vm-brep@.service /etc/systemd/system/
+# chmod 644 /etc/systemd/system/vm-brep@.service
+# systemctl status vm-brep@brep
+# systemctl start vm-brep@brep
+# systemctl status vm-brep@brep
+
+If the VM fails to start, study the log for a possible cause:
+
+# journalctl -u vm-brep@brep
+
+If the VM has started successfully, perform the same verifications as on step
+4 above.
+
+To login to the VM running as a systemd service (for example, to verify IP
+and hostname) use the vm-login script (which uses screen to connect to the
+VM's console):
+
+$ ~/bin/vm-login ~/brep-con.sock
+
+To close the login, press Ctrl-a k (or Ctrl-a a k if already running inside
+screen).
+
+If everything functions correctly, verify the VM can be stopped:
+
+# systemctl stop vm-brep@brep
+# systemctl status vm-brep@brep
+
+Finally, if desired, enable the VM service to start on boot:
+
+# systemctl enable vm-brep@brep
+
+After this you may also want to reboot the host machine and confirm the VM is
+started on boot.
+
+
+6. Managing the repository state.
+
+While you can submit packages to the repository using bdep-publish(1), you can
+also add them manually. Also, packages can only be removed manually.
+
+The repository packages and metadata are stored in the ~brep/state/bpkg/pkg/
+directory. If you need to make any modifications in this directory, there are
+two rules that you must follow:
+
+  1. You must stop the VM before making any modifications.
+
+  2. You must make any modification only as user brep.
+
+After performing the modifications, remove the 1/packages.manifest file to
+trigger the repository metadata regeneration on the next VM startup.
+
+Putting it all together, the steps could look like this:
+
+# systemctl stop vm-brep@brep
+# su - brep
+$ cd state/bpkg/pkg
+$ ...
+$ rm 1/packages.manifest
+$ exit
+# systemctl start vm-brep@brep
+
+Note also that it's easy to break the repository with manual modifications.
+For example, you may add a package that has an unmet dependency or remove a
+package that still has some dependents. In this case, the repository web
+interface will be unavailable. As a result, you may want to login into the VM
+and verify the repository regeneration was successful:
+
+$ ~/bin/vm-login ~/brep-con.sock
+
+  # systemctl status brep-startup
+  # journalctl -u brep-startup