Initial manifest specification

1 files changed, 304 insertions, 3 deletions

diff --git a/doc/manual.cli b/doc/manual.cli
index 9d0d4f2..49cf8d9 100644
--- a/doc/manual.cli
+++ b/doc/manual.cli
@@ -49,7 +49,7 @@ server.
 
 Once an agent receives a build task for a specific build machine, it goes
 through the following steps. First, it creates a directory on its TFTP server
-with the \i{build machine id} as its name and places the build task manifest
+with the \i{machine name} as its name and places the build task manifest
 inside. Next, it makes a throw-away snapshot of the build machine and boots
 it. After booting the build machine, the agent monitors the machine directory
 on its TFTP server for the build result manifest (uploaded by the worker once
@@ -65,8 +65,9 @@ to use.
 If the controller has higher-level controllers (for example, \c{brep}), then
 it aggregates the available build machines from its agents and polls these
 controllers (just as an agent would), forwarding build tasks to suitable
-agents. The controller may also be configured to monitor build sources, such
-as SCM repositories, directly in which case it generates build tasks itself.
+agents. In this case we say that the \i{controller act as an agent}. The
+controller may also be configured to monitor build sources, such as SCM
+repositories, directly in which case it generates build tasks itself.
 
 In this architecture the build results are propagated up the chain: from a
 worker, to its agent, to its controller, and so on. A controller that is the
@@ -76,4 +77,304 @@ owner if the build failed. Similarly, a \c{bbot} controller that monitors a
 \c{git} repository would send an email to a committer if their commit caused a
 build failure. The email would include a link (normally HTTP/HTTPS) to the
 build logs hosted by the controller.
+
+\h2#arch-machine-config|Machine Configuration|
+
+A build machine has a notion of a \i{machine configuration} that captures the
+operating system, installed compiler toolchain, and so on. Note that the same
+build machine may be used for multiple \i{build configurations}. For example,
+the same machine can normally be used to produce 32/64-bit and debug/release
+builds.
+
+The machine configuration is \i{approximately} encoded in its \i{machine
+name}. The machine name is a list of components separated with \c{-}. Each
+component can contain alpha-numeric characters, underscores, dots, and pluses
+with the whole id being a portably-valid path component.
+
+The encoding is approximate in a sense that it captures only what's important
+to distinguish in a particular \c{bbot} deployment.
+
+The first component normally identifies the operating system and has the
+following recommended form:
+
+\
+[<arch>_]<os>[_<version>]
+\
+
+For example:
+
+\
+windows
+windows_10
+windows_10.1607
+i686_windows_xp
+freebsd_10
+centos_6.2
+ubuntu_16.04
+macos_10.12
+\
+
+The second component normally identifies the installed compiler toolchain and
+has the following recommended form:
+
+\
+<id>[<version>][<runtime>]
+\
+
+For example:
+
+\
+gcc
+gcc_6
+gcc_6.3
+clang_3.9_libc++
+clang_3.9_libstdc++
+msvc_14
+msvc_14u3
+icc
+\
+
+Some examples of complete machine names:
+
+\
+windows_10-msvc_14u3
+macos_10.12-clang
+ubuntu_16.04-gcc_6.3
+\
+
+\h2#arch-machine-manifest|Machine Manifest|
+
+The build machine manifest describes the build machine on the build host (see
+the Build OS documentation for their origin and location). A list of machine
+manifests is also sent by \c{bbot} agents to controllers.
+
+\
+SYNOPSIS
+
+id:      <machine-id>
+name:    <machine-name>
+type:    <machine-type>
+summary: <string>
+\
+
+\dl|
+
+\li|\n\c{id: <machine-id>}\n
+
+  The \i{machine-id} uniquely identifies a machine version/revision/build.
+  For virtual machines this can be the disk image checksum. For a container
+  this can be UUID that is re-generated every time a container filesystem
+  is altered.|
+
+\li|\n\c{name: <machine-name>}\n
+
+  The machine name as described above.|
+
+\li|\n\c{type: <machine-type>}\n
+
+  The machine type. Valid values are \c{vm} and \c{container}. Note that this
+  value is not sent by agents to controllers.|
+
+\li|\n\c{summary: <string>}\n
+
+  A one-line description of the machine. For example:
+
+  \
+  name: windows_10-msvc_14
+  summary: Windows 10 build 1607 with VC 14 update 3
+  \
+
+||
+
+
+\h2#arch-task-manifest|Task Manifest|
+
+The task manifest describes a build task.
+
+\
+SYNOPSIS
+
+name:       <package-name>
+version:    <package-version>
+repository: <repository-url>
+#location:  <package-url>
+\
+
+\dl|
+
+\li|\n\c{name: <package-name>}\n
+
+  Package name to test.|
+
+\li|\n\c{version: <package-version>}\n
+
+  Package version to test.|
+
+\li|\n\c{repository: <repository-url>}\n
+
+  The \c{bpkg} repository that contains the package and its dependencies.||
+
+To test a package from a repository the \c{bbot} worker would execute
+the following commands:
+
+\
+bpkg -v create [cxx config.cxx=g++-6]
+bpkg -v add <repository-url>
+bpkg -v fetch
+bpkg -v build --yes --configure-only <package-name>/<package-version>
+bpkg -v update <package-name>
+bpkg -v test <package-name>
+\
+
+\h2#arch-result-manifest|Result Manifest|
+
+The result manifest describes a build result.
+
+\
+SYNOPSIS
+
+name:     <package-name>
+version:  <package-version>
+
+status:           <status>
+configure-status: <status>
+update-status:    <status>
+test-status:      <status>
+
+configure-log: <text>
+update-log:    <text>
+test-log:      <text>
+\
+
+\dl|
+
+\li|\n\c{name: <package-name>}\n
+
+  Package name from the task manifest.|
+
+\li|\n\c{version: <package-version>}\n
+
+  Package version from the task manifest.|
+
+\li|\n\c{status: <status>}\n
+
+  An overall (cumulative) build result status. Valid values are:
+
+  \
+  success    # All operations completed successfully.
+  warning    # One or more operations completed with warnings.
+  error      # One or more operations completed with errors.
+  abort      # One or more operations were aborted.
+  abnormal   # One or more operations terminated abnormally.
+  \
+
+  The \c{abort} status indicates that the operation has been aborted by
+  \c{bbot}, for example, because it was consuming too many resources and/or
+  was taking too long. Note that a task can be aborted both by the \c{bbot}
+  worker as well as the agent. In the later case the whole machine is shut
+  down and no operation-specific status or logs will be included (@@ Maybe
+  we should just include 'log:' with commands that start VM, for
+  completeness?).
+
+  The \c{abnormal} status indicates that the operation has terminated
+  abnormally, for example, due to the package manager or build system crash.
+
+  Note that the overall \c{status} value should appear before any
+  per-operation \c{*-status} values.|
+
+\li|\n\c{*-status: <status>}\n
+
+  A per-operation result status. Note that the \c{*-status} values should
+  appear in the same order as the corresponding operations were performed
+  and for each \c{*-status} there should be a corresponding \c{*-log}.|
+
+\li|\n\c{*-log: <text>}\n
+
+  A per-operation result log. Note that the \c{*-log} values should appear
+  last and in the same order as the corresponding \c{*-status} values.||
+
+
+\h2#arch-task-req-manifest|Task Request Manifest|
+
+An agent (or controller acting as an agent) sends a task request to its
+controller via HTTP/HTTPS POST method (@@ URL/API endpoint). The task request
+starts with the task request manifest followed by a list of machine manifests.
+
+\
+SYNOPSIS
+
+agent:       <agent-name>
+fingerprint: <agent-fingerprint>
+\
+
+\dl|
+
+\li|\n\c{agent: <agent-name>}\n
+
+  The name of the agent host (\c{hostname}). These should be unique in a
+  particular \c{bbot} deployment.|
+
+\li|\n\c{fingerprint: <agent-fingerprint>}\n
+
+  The SHA256 fingerprint of the agent's public key.||
+
+
+\h2#arch-task-res-manifest|Task Response Manifest|
+
+A controller sends the task response manifest in response to the task request
+initiated by an agent. The response is delivered as a result of the POST
+method. The task response starts with the task response manifest optionally
+followed by a task manifest.
+
+\
+SYNOPSIS
+
+session:   <session-id>
+challenge: <text>
+\
+
+\dl|
+
+\li|\n\c{session: <session-id>}\n
+
+  An identifier assigned to this session by the controller. An empty value
+  indicates that the controller has no tasks at this time in which case the
+  task manifest is absent.|
+
+\li|\n\c{challenge: <text>}\n
+
+  A random text (nonce) used to challenge the agent's private key. If present,
+  then the agent must sign this text with its private key and include the
+  signature in the result request.
+
+  The signature should be derived by calculating the SHA256 checksum of the
+  text, encrypting it with the agent's private key, and then base64-encoding
+  the result.||
+
+
+\h2#arch-result-req-manifest|Result Request Manifest|
+
+On completion of a task an agent (or controller acting as an agent) sends a
+result (upload) request to its controller via HTTP/HTTPS POST method (@@
+URL/API endpoint). The result request starts with the result request manifest
+followed by a result manifest. Note that there is no result response and
+only a successful but empty POST result is returned.
+
+\
+SYNOPSIS
+
+session:   <session-id>
+challenge: <text>
+\
+
+\dl|
+
+\li|\n\c{session: <session-id>}\n
+
+  The session id as returned by the controller in the task response.|
+
+\li|\n\c{challenge: <text>}\n
+
+  The answer to the private key challenge as posed by the controller in the
+  task response.||
 "