Implement brep-clean

21 files changed, 715 insertions, 61 deletions

diff --git a/INSTALL b/INSTALL
index f6e967d..14e9ae9 100644
--- a/INSTALL
+++ b/INSTALL
@@ -6,8 +6,9 @@ replace systemctl commands with the equivalent init.d ones.
 
 1. Create 'brep' User
 
-This user will be used to run the brep repository loader. We will also use its
-home directory to build and install the brep module, store its configuration,
+This user will be used to run the brep package database loader, build database
+cleaner, and the database schemes migration utility. We will also use its home
+directory to build and install the brep module, store its configuration,
 etc. We create this user with a disabled password so only root will be able to
 operate as brep. Because of this restriction we will allow brep to run sudo
 without a password:
@@ -255,36 +256,42 @@ $ cd install/share/brep/www/
 $ for i in *.scss; do sassc -s compressed $i `basename -s .scss $i`.css; done
 
 
-8. Setup Periodic Loader Execution
+8. Setup Periodic Loader and Cleaner Execution
 
 Initially this guide suggested using systemd user session support to run the
-loader. However, the current state of user sessions has one major drawback:
-they are not started/attached-to when logging in with su -l (see Debian bug
-#813789 for details). This limitation makes them unusable in our setup. If you
-still would like to use systemd to run the loader, then you can set it up as a
-system-wide service which runs the loader as the brep user/group. Otherwise, a
-cron job is a natural choice.
+loader and the cleaner. However, the current state of user sessions has one
+major drawback: they are not started/attached-to when logging in with su -l
+(see Debian bug #813789 for details). This limitation makes them unusable in
+our setup. If you still would like to use systemd to run the loader and the
+cleaner, then you can set it up as a system-wide service which runs the
+utilities as the brep user/group. Otherwise, a cron job is a natural choice.
 
+Note that the cleaner execution is optional and is only required if the build2
+build bot functionality is enabled (see the build bot documentation for
+details). If it is disabled in you setup, then skip the cleaner-related
+parts in the subsequent subsections.
 
-8.a Setup Periodic Loader Execution with cron
+8.a Setup Periodic Loader and Cleaner Execution with cron
 
-The following crontab entry will execute the loader every five minutes:
+The following crontab entries will execute the loader every five minutes
+and the cleaner once a day at midnight:
 
 $ crontab -l
 MAILTO=<brep-admin-email>
 PATH=/usr/local/bin:/bin:/usr/bin
 */5 * * * * $HOME/install/bin/brep-load $HOME/config/loadtab
+0 0 * * * $HOME/install/bin/brep-clean $HOME/config/buildtab
 ^D
 
 Note that here we assume that bpkg (which is executed by brep-load) is in one
 of the PATH's directories (usually /usr/local/bin).
 
 
-8.b Setup Periodic Loader Execution with systemd
+8.b Setup Periodic Loader and Cleaner Execution with systemd
 
 In this version we will use the systemd user session to periodically run the
-loader as the brep user. If your installation doesn't use systemd, then a cron
-job would be a natural alternative (see above).
+loader and the cleaner as the brep user. If your installation doesn't use
+systemd, then a cron job would be a natural alternative (see above).
 
 As the first step, make sure systemd user sessions support is working for the
 brep user:
@@ -296,26 +303,32 @@ installed, relogin as brep, and try again. If it still doesn't work, google
 for the error message and your distribution name.
 
 Next enable the brep's systemd session to remain running after logging off
-since we want the loader to run even when we are not logged in:
+since we want the utilities to run even when we are not logged in:
 
 $ sudo loginctl enable-linger brep
 
 $ mkdir -p .config/systemd/user
 $ cp install/share/brep/etc/systemd/brep-load.* .config/systemd/user/
+$ cp install/share/brep/etc/systemd/brep-clean.* .config/systemd/user/
 
 Start the service to make sure there are no issues:
 
 $ systemctl --user start brep-load.service
 $ journalctl
 
-Start the timer and monitor it to make sure it fires:
+$ systemctl --user start brep-clean.service
+$ journalctl
+
+Start the timers and monitor them to make sure they fire:
 
 $ systemctl --user start brep-load.timer
+$ systemctl --user start brep-clean.timer
 $ journalctl -f
 
 If everything looks good, enable the timer to be started at boot time:
 
 $ systemctl --user enable brep-load.timer
+$ systemctl --user enable brep-clean.timer
 
 
 9. Upgrade Procedure
@@ -337,15 +350,18 @@ $ cd brep
 $ bpkg fetch
 $ bpkg build brep
 
-If you are using a systemd-based setup, then stop and disable the loader:
+If you are using a systemd-based setup, then stop and disable the loader and
+the cleaner:
 
 $ systemctl --user disable --now brep-load.timer
+$ systemctl --user disable --now brep-clean.timer
 $ systemctl --user stop brep-load.service
+$ systemctl --user stop brep-clean.service
 
 If you are using a cron-based setup, then it is not worth it commenting out the
-job entry. If the new version of the loader gets executed before or during the
-migration, then it will fail and you will get an email with the diagnostics.
-Other than that, it should be harmless.
+job entries. If the new version of the loader or the cleaner gets executed
+before or during the migration, then it will fail and you will get an email
+with the diagnostics. Other than that, it should be harmless.
 
 Stop apache:
 
@@ -372,14 +388,17 @@ is not possible), then one way to do it would be:
 $ psql -d brep_package -c 'DROP OWNED BY brep'
 $ psql -d brep_build -c 'DROP OWNED BY brep'
 
-If using systemd, then start and enable the loader:
+If using systemd, then start and enable the loader and the cleaner:
 
 $ systemctl --user start brep-load.service
 $ systemctl --user status brep-load.service
+$ systemctl --user start brep-clean.service
+$ systemctl --user status brep-clean.service
 
 If everything looks good, enable periodic execution:
 
 $ systemctl --user enable --now brep-load.timer
+$ systemctl --user enable --now brep-clean.timer
 
 If using cron, then simply wait for the next run.
 
diff --git a/buildfile b/buildfile
index 25d38ea..8c80253 100644
--- a/buildfile
+++ b/buildfile
@@ -2,7 +2,7 @@
 # copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
 # license   : MIT; see accompanying LICENSE file
 
-./: libbrep/ mod/ load/ migrate/ tests/ www/ doc/ etc/ \
+./: libbrep/ mod/ clean/ load/ migrate/ tests/ www/ doc/ etc/ \
   doc{INSTALL INSTALL-DEV LICENSE NEWS README version} file{manifest}
 
 doc{version}: file{manifest} # Generated by the version module.
diff --git a/clean/.gitignore b/clean/.gitignore
new file mode 100644
index 0000000..8e11bbd
--- /dev/null
+++ b/clean/.gitignore
@@ -0,0 +1,2 @@
+*-options.?xx
+brep-clean
diff --git a/clean/buildfile b/clean/buildfile
new file mode 100644
index 0000000..b4feada
--- /dev/null
+++ b/clean/buildfile
@@ -0,0 +1,30 @@
+# file      : clean/buildfile
+# copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+# license   : MIT; see accompanying LICENSE file
+
+import libs  = libodb%lib{odb}
+import libs += libodb-pgsql%lib{odb-pgsql}
+import libs += libbutl%lib{butl}
+import libs += libbbot%lib{bbot}
+
+include ../libbrep/
+
+exe{brep-clean}: {hxx ixx cxx}{* -clean-options} \
+                 {hxx ixx cxx}{clean-options}    \
+                 ../libbrep/lib{brep} $libs
+
+# Generated options parser.
+#
+if $cli.configured
+{
+  cli.cxx{clean-options}: cli{clean}
+
+  cli.options += -I $src_root --include-with-brackets --include-prefix clean \
+--guard-prefix CLEAN --generate-specifier --page-usage print_ --ansi-color   \
+--cxx-prologue "#include <clean/types-parsers.hxx>"                          \
+--long-usage
+
+  # Include generated cli files into the distribution.
+  #
+  cli.cxx{clean-options}: dist = true
+}
diff --git a/clean/clean.cli b/clean/clean.cli
new file mode 100644
index 0000000..3f4e2ea
--- /dev/null
+++ b/clean/clean.cli
@@ -0,0 +1,164 @@
+// file      : clean/clean.cli
+// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+// license   : MIT; see accompanying LICENSE file
+
+include <vector>;
+include <string>;
+include <cstdint>; // uint16_t
+
+include <clean/options-types.hxx>;
+
+"\section=1"
+"\name=brep-clean"
+"\summary=clean brep build database"
+
+{
+  "<options> <buildtab>",
+
+  "\h|SYNOPSIS|
+
+   \cb{brep-clean --help}\n
+   \cb{brep-clean --version}\n
+   \c{\b{brep-clean} [<options>] <buildtab>}
+
+   \h|DESCRIPTION|
+
+   \cb{brep-clean} deletes expired package builds from the brep \cb{build}
+   database. The build is considered expired if the package version is not
+   in the \cb{package} database, or the configuration is not listed in the
+   <buildtab> file, or the timestamp is older than the one specified for
+   this build's toolchain (see \cb{--stale-timeout}).
+
+   Note that \cb{brep-clean} expects the \cb{build} and \cb{package} database
+   schemas to have already been created using \l{brep-migrate(1)}."
+}
+
+class options
+{
+  "\h|OPTIONS|"
+
+  brep::toolchain_timeouts --stale-timeout
+  {
+    "[<name>=]<days>",
+    "Number of days to wait before considering builds for the named toolchain
+     as stale. Specify zero for <days> to make builds for a toolchain never
+     expire. Omit <name> (including \cb{=}) to specify the default timeout.
+     It will apply to all the toolchains that don't have a timeout specified
+     explicitly. If unspecified, the default timeout is zero (never expire)."
+  }
+
+  std::string --build-db-user
+  {
+    "<user>",
+    "Build database user name. If not specified, then operating system (login)
+     name is used."
+  }
+
+  std::string --build-db-password
+  {
+    "<pass>",
+    "Build database password. If not specified, then login without password is
+     expected to work."
+  }
+
+  std::string --build-db-name = "brep_build"
+  {
+    "<name>",
+    "Build database name. If not specified, then \cb{brep_build} is used by
+     default."
+  }
+
+  std::string --build-db-host
+  {
+    "<host>",
+    "Build database host name, address, or socket. If not specified, then
+     connect to \cb{localhost} using the operating system-default mechanism
+     (Unix-domain socket, etc)."
+  }
+
+  std::uint16_t --build-db-port = 0
+  {
+    "<port>",
+    "Build database port number. If not specified, the default port is used."
+  }
+
+  std::string --package-db-user
+  {
+    "<user>",
+    "Package database user name. If not specified, then operating system
+     (login) name is used."
+  }
+
+  std::string --package-db-password
+  {
+    "<pass>",
+    "Package database password. If not specified, then login without password
+     is expected to work."
+  }
+
+  std::string --package-db-name = "brep_package"
+  {
+    "<name>",
+    "Package database name. If not specified, then \cb{brep_package} is used by
+     default."
+  }
+
+  std::string --package-db-host
+  {
+    "<host>",
+    "Package database host name, address, or socket. If not specified, then
+     connect to \cb{localhost} using the operating system-default mechanism
+     (Unix-domain socket, etc)."
+  }
+
+  std::uint16_t --package-db-port = 0
+  {
+    "<port>",
+    "Package database port number. If not specified, the default port is used."
+  }
+
+  std::string --pager // String to allow empty value.
+  {
+    "<path>",
+    "The pager program to be used to show long text. Commonly used pager
+     programs are \cb{less} and \cb{more}. You can also specify additional
+     options that should be passed to the pager program with
+     \cb{--pager-option}. If an empty string is specified as the pager
+     program, then no pager will be used. If the pager program is not
+     explicitly specified, then \cb{brep-clean} will try to use \cb{less}.
+     If it is not available, then no pager will be used."
+  }
+
+  std::vector<std::string> --pager-option
+  {
+    "<opt>",
+    "Additional option to be passed to the pager program. See \cb{--pager}
+     for more information on the pager program. Repeat this option to
+     specify multiple pager options."
+  }
+
+  bool --help {"Print usage information and exit."}
+  bool --version {"Print version and exit."}
+};
+
+"\h|EXIT STATUS|
+
+\dl|
+
+\li|\cb{0}
+
+Success.|
+
+\li|\cb{1}
+
+Fatal error.|
+
+\li|\cb{2}
+
+An instance of \cb{brep-clean} or \l{brep-migrate(1)} is already running. Try
+again.|
+
+\li|\cb{3}
+
+Recoverable database error. Try again.||
+"
diff --git a/clean/clean.cxx b/clean/clean.cxx
new file mode 100644
index 0000000..fbb4d23
--- /dev/null
+++ b/clean/clean.cxx
@@ -0,0 +1,287 @@
+// file      : clean/clean.cxx -*- C++ -*-
+// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+// license   : MIT; see accompanying LICENSE file
+
+#include <set>
+#include <iostream>
+
+#include <odb/database.hxx>
+#include <odb/transaction.hxx>
+#include <odb/schema-catalog.hxx>
+
+#include <odb/pgsql/database.hxx>
+
+#include <libbutl/pager.hxx>
+
+#include <libbbot/build-config.hxx>
+
+#include <libbrep/build.hxx>
+#include <libbrep/build-odb.hxx>
+#include <libbrep/package.hxx>
+#include <libbrep/package-odb.hxx>
+#include <libbrep/database-lock.hxx>
+
+#include <clean/clean-options.hxx>
+
+using namespace std;
+using namespace bbot;
+using namespace brep;
+using namespace odb::core;
+
+// Operation failed, diagnostics has already been issued.
+//
+struct failed {};
+
+static const char* help_info (
+  "  info: run 'brep-clean --help' for more information");
+
+int
+main (int argc, char* argv[])
+try
+{
+  cli::argv_scanner scan (argc, argv, true);
+  options ops (scan);
+
+  // Version.
+  //
+  if (ops.version ())
+  {
+    cout << "brep-clean " << BREP_VERSION_ID << endl
+         << "libbrep " << LIBBREP_VERSION_ID << endl
+         << "libbbot " << LIBBBOT_VERSION_ID << endl
+         << "libbpkg " << LIBBPKG_VERSION_ID << endl
+         << "libbutl " << LIBBUTL_VERSION_ID << endl
+         << "Copyright (c) 2014-2017 Code Synthesis Ltd" << endl
+         << "This is free software released under the MIT license." << endl;
+
+    return 0;
+  }
+
+  // Help.
+  //
+  if (ops.help ())
+  {
+    butl::pager p ("brep-clean help",
+                   false,
+                   ops.pager_specified () ? &ops.pager () : nullptr,
+                   &ops.pager_option ());
+
+    print_usage (p.stream ());
+
+    // If the pager failed, assume it has issued some diagnostics.
+    //
+    return p.wait () ? 0 : 1;
+  }
+
+  const toolchain_timeouts& timeouts (ops.stale_timeout ());
+
+  auto i (timeouts.find (string ()));
+  timestamp default_timeout (i != timeouts.end ()
+                             ? i->second
+                             : timestamp_nonexistent);
+
+  // Load configurations names.
+  //
+  if (!scan.more ())
+  {
+    cerr << "error: configuration file expected" << endl
+         << help_info << endl;
+    return 1;
+  }
+
+  set<string> configs;
+  for (auto& c: parse_buildtab (path (scan.next ())))
+    configs.emplace (move (c.name));
+
+  if (scan.more ())
+  {
+    cerr << "error: unexpected argument encountered" << endl
+         << help_info << endl;
+    return 1;
+  }
+
+  odb::pgsql::database build_db (
+    ops.build_db_user (),
+    ops.build_db_password (),
+    ops.build_db_name (),
+    ops.build_db_host (),
+    ops.build_db_port (),
+    "options='-c default_transaction_isolation=serializable'");
+
+  odb::pgsql::database package_db (
+    ops.package_db_user (),
+    ops.package_db_password (),
+    ops.package_db_name (),
+    ops.package_db_host (),
+    ops.package_db_port (),
+    "options='-c default_transaction_isolation=serializable'");
+
+  // Prevent several brep-clean/migrate instances from updating build database
+  // simultaneously.
+  //
+  database_lock l (build_db);
+
+  // Check that the build and package database schemas match the current ones.
+  //
+  const string bs ("build");
+  if (schema_catalog::current_version (build_db, bs) !=
+      build_db.schema_version (bs))
+  {
+    cerr << "error: build database schema differs from the current one"
+         << endl << "  info: use brep-migrate to migrate the database" << endl;
+    return 1;
+  }
+
+  const string ps ("package");
+  if (schema_catalog::current_version (package_db, ps) !=
+      package_db.schema_version (ps))
+  {
+    cerr << "error: package database schema differs from the current one"
+         << endl << "  info: use brep-migrate to migrate the database" << endl;
+    return 1;
+  }
+
+  // Prepare the build prepared query.
+  //
+  // Query package builds in chunks in order not to hold locks for too long.
+  // Sort the result by package version to minimize number of queries to the
+  // package database.
+  //
+  using bld_query = query<build>;
+  using prep_bld_query = prepared_query<build>;
+
+  size_t offset (0);
+  bld_query bq ("ORDER BY" + bld_query::id.package.name +
+                order_by_version_desc (bld_query::id.package.version, false) +
+                "OFFSET" + bld_query::_ref (offset) + "LIMIT 100");
+
+  connection_ptr bld_conn (build_db.connection ());
+
+  prep_bld_query bld_prep_query (
+    bld_conn->prepare_query<build> ("build-query", bq));
+
+  // Prepare the package version query.
+  //
+  // Query package versions every time the new package name is encountered
+  // during iterating over the package builds. Such a query will be made once
+  // per package name due to the builds query sorting criteria (see above).
+  //
+  using pkg_query = query<package_version>;
+  using prep_pkg_query = prepared_query<package_version>;
+
+  string package_name;
+  set<version> package_versions;
+
+  pkg_query pq (pkg_query::package::id.name == pkg_query::_ref (package_name));
+
+  connection_ptr pkg_conn (package_db.connection ());
+
+  prep_pkg_query pkg_prep_query (
+    pkg_conn->prepare_query<package_version> ("package-version-query", pq));
+
+  while (true)
+  {
+    // Start the build database transaction.
+    //
+    transaction bt (bld_conn->begin ());
+
+    // Query builds.
+    //
+    auto builds (bld_prep_query.execute ());
+
+    if (!builds.empty ())
+    {
+      // Start the package database transaction.
+      //
+      transaction pt (pkg_conn->begin (), false);
+
+      for (const auto& b: builds)
+      {
+        auto i (timeouts.find (b.toolchain_name));
+
+        timestamp et (i != timeouts.end ()
+                      ? i->second
+                      : default_timeout);
+
+        bool cleanup (
+          // Check that the build is not stale.
+          //
+          b.timestamp <= et ||
+
+          // Check that the build configuration is still present.
+          //
+          // Note that we unable to detect configuration changes and rely on
+          // periodic rebuilds to take care of that.
+          //
+          configs.find (b.configuration) == configs.end ());
+
+        // Check that the build package still exists.
+        //
+        if (!cleanup)
+        {
+          if (package_name != b.package_name)
+          {
+            // Switch to the package database transaction.
+            //
+            transaction::current (pt);
+
+            package_name = b.package_name;
+            package_versions.clear ();
+
+            for (auto& v: pkg_prep_query.execute ())
+              package_versions.emplace (move (v.version));
+
+            // Switch back to the build database transaction.
+            //
+            transaction::current (bt);
+          }
+
+          cleanup = package_versions.find (b.package_version) ==
+            package_versions.end ();
+        }
+
+        if (cleanup)
+          build_db.erase (b);
+        else
+          ++offset;
+      }
+
+      // Commit the package database transaction.
+      //
+      pt.commit ();
+    }
+
+    bt.commit ();
+
+    if (builds.empty ())
+      break;
+  }
+
+  return 0;
+}
+catch (const database_locked&)
+{
+  cerr << "brep-clean or brep-migrate is running" << endl;
+  return 2;
+}
+catch (const recoverable& e)
+{
+  cerr << "recoverable database error: " << e << endl;
+  return 3;
+}
+catch (const cli::exception& e)
+{
+  cerr << "error: " << e << endl << help_info << endl;
+  return 1;
+}
+catch (const failed&)
+{
+  return 1; // Diagnostics has already been issued.
+}
+// Fully qualified to avoid ambiguity with odb exception.
+//
+catch (const std::exception& e)
+{
+  cerr << "error: " << e << endl;
+  return 1;
+}
diff --git a/clean/options-types.hxx b/clean/options-types.hxx
new file mode 100644
index 0000000..183c0df
--- /dev/null
+++ b/clean/options-types.hxx
@@ -0,0 +1,18 @@
+// file      : clean/options-types.hxx -*- C++ -*-
+// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+// license   : MIT; see accompanying LICENSE file
+
+#ifndef CLEAN_OPTIONS_TYPES_HXX
+#define CLEAN_OPTIONS_TYPES_HXX
+
+#include <map>
+
+#include <libbrep/types.hxx>
+#include <libbrep/utility.hxx>
+
+namespace brep
+{
+  struct toolchain_timeouts: std::map<string, timestamp> {};
+}
+
+#endif // CLEAN_OPTIONS_TYPES_HXX
diff --git a/clean/types-parsers.cxx b/clean/types-parsers.cxx
new file mode 100644
index 0000000..26b9a02
--- /dev/null
+++ b/clean/types-parsers.cxx
@@ -0,0 +1,60 @@
+// file      : clean/types-parsers.cxx -*- C++ -*-
+// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+// license   : MIT; see accompanying LICENSE file
+
+#include <chrono>
+#include <string> // strtoull()
+
+#include <clean/types-parsers.hxx>
+
+#include <clean/options-types.hxx>
+#include <clean/clean-options.hxx> // cli namespace
+
+using namespace std;
+using namespace brep;
+
+namespace cli
+{
+  void parser<toolchain_timeouts>::
+  parse (toolchain_timeouts& x, bool& xs, scanner& s)
+  {
+    const char* o (s.next ());
+
+    if (!s.more ())
+      throw missing_value (o);
+
+    string ov (s.next ());
+    size_t p (ov.find ('='));
+
+    timestamp now (timestamp::clock::now ());
+
+    // Convert timeout duration into the time point.
+    //
+    auto timeout = [o, &ov, &now] (const string& tm) -> timestamp
+    {
+      char* e (nullptr);
+      uint64_t t (strtoull (tm.c_str (), &e, 10));
+
+      if (*e != '\0' || tm.empty ())
+        throw invalid_value (o, ov);
+
+      if (t == 0)
+        return timestamp_nonexistent;
+
+      return now - chrono::duration<uint64_t, ratio<86400>> (t);
+    };
+
+    if (p == string::npos)
+      x[string ()] = timeout (ov); // Default timeout.
+    else
+    {
+      string k (ov, 0, p);
+      if (k.empty ())
+        throw invalid_value (o, ov);
+
+      x[k] = timeout (string (ov, p + 1));
+    }
+
+    xs = true;
+  }
+}
diff --git a/clean/types-parsers.hxx b/clean/types-parsers.hxx
new file mode 100644
index 0000000..46fd483
--- /dev/null
+++ b/clean/types-parsers.hxx
@@ -0,0 +1,28 @@
+// file      : clean/types-parsers.hxx -*- C++ -*-
+// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd
+// license   : MIT; see accompanying LICENSE file
+
+// CLI parsers, included into the generated source files.
+//
+
+#ifndef CLEAN_TYPES_PARSERS_HXX
+#define CLEAN_TYPES_PARSERS_HXX
+
+#include <clean/options-types.hxx>
+
+namespace cli
+{
+  class scanner;
+
+  template <typename T>
+  struct parser;
+
+  template <>
+  struct parser<brep::toolchain_timeouts>
+  {
+    static void
+    parse (brep::toolchain_timeouts&, bool&, scanner&);
+  };
+}
+
+#endif // CLEAN_TYPES_PARSERS_HXX
diff --git a/doc/buildfile b/doc/buildfile
index 31b0dd6..278552f 100644
--- a/doc/buildfile
+++ b/doc/buildfile
@@ -3,7 +3,8 @@
 # license   : MIT; see accompanying LICENSE file
 
 cmds =             \
-brep-load 	   \
+brep-clean         \
+brep-load          \
 brep-migrate
 
 define css: file
diff --git a/doc/cli.sh b/doc/cli.sh
index 4304138..6d3d09c 100755
--- a/doc/cli.sh
+++ b/doc/cli.sh
@@ -53,7 +53,7 @@ o="--output-prefix brep-"
 #
 #compile "brep" $o --output-prefix ""
 
-pages="load/load migrate/migrate"
+pages="clean/clean load/load migrate/migrate"
 
 for p in $pages; do
   compile $p $o
diff --git a/etc/systemd/brep-clean.service b/etc/systemd/brep-clean.service
new file mode 100644
index 0000000..0099d62
--- /dev/null
+++ b/etc/systemd/brep-clean.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=brep build database cleaner service
+
+[Service]
+Type=oneshot
+#User=brep
+#Group=brep
+ExecStart=/home/brep/install/bin/brep-clean /home/brep/config/buildtab
+
+[Install]
+WantedBy=default.target
diff --git a/etc/systemd/brep-clean.timer b/etc/systemd/brep-clean.timer
new file mode 100644
index 0000000..f4c587e
--- /dev/null
+++ b/etc/systemd/brep-clean.timer
@@ -0,0 +1,23 @@
+[Unit]
+Description=brep build database cleaner timer
+RefuseManualStart=no
+RefuseManualStop=no
+
+[Timer]
+Unit=brep-clean.service
+
+# Don't keep track of the timer across reboots.
+#
+Persistent=false
+
+# Wait 20 seconds until the first run.
+#
+OnBootSec=20
+
+# Then wait 5 minutes until the next run.
+#
+OnUnitInactiveSec=5m
+
+
+[Install]
+WantedBy=timers.target
diff --git a/load/.gitignore b/load/.gitignore
index 49afbe2..035e847 100644
--- a/load/.gitignore
+++ b/load/.gitignore
@@ -1,3 +1,2 @@
-*-options
 *-options.?xx
 brep-load
diff --git a/load/load.cli b/load/load.cli
index 51a34c3..826d997 100644
--- a/load/load.cli
+++ b/load/load.cli
@@ -10,25 +10,25 @@ include <libbrep/types.hxx>;
 
 "\section=1"
 "\name=brep-load"
-"\summary=load build2 repositories into database"
+"\summary=load build2 repositories into brep package database"
 
 {
-  "<options> <loadtab-file>",
+  "<options> <loadtab>",
 
   "\h|SYNOPSIS|
 
    \cb{brep-load --help}\n
    \cb{brep-load --version}\n
-   \c{\b{brep-load} [<options>] <loadtab-file>}
+   \c{\b{brep-load} [<options>] <loadtab>}
 
    \h|DESCRIPTION|
 
    \cb{brep-load} reads the list of repositories from the specified
-   configuration <loadtab-file>, fetches their manifest files, and loads the
-   repository and package information into the database, suitable for
-   consumption by the \cb{brep} web module.
+   <loadtab> configuration file, fetches their manifest files, and loads the
+   repository and package information into the \cb{package} database, suitable
+   for consumption by the \cb{brep} web module.
 
-   Note that \cb{brep-load} expects the database \cb{package} schema to have
+   Note that \cb{brep-load} expects the \cb{package} database schema to have
    already been created using \l{brep-migrate(1)}.
 
    Also note that \cb{brep-load} requires \l{bpkg(1)} to fetch repository
@@ -119,12 +119,22 @@ class options
 
 "\h|EXIT STATUS|
 
-\cb{0}      Successful termination.
+\dl|
 
-\cb{1}      Fatal error.
+\li|\cb{0}
 
-\cb{2}      \cb{brep-load} or \l{brep-migrate(1)} instance is running. Try
-            again.
+Success.|
 
-\cb{3}      The database recoverable error. Try again.
+\li|\cb{1}
+
+Fatal error.|
+
+\li|\cb{2}
+
+An instance of \cb{brep-load} or \l{brep-migrate(1)} is already running. Try
+again.|
+
+\li|\cb{3}
+
+Recoverable database error. Try again.||
 "
diff --git a/load/load.cxx b/load/load.cxx
index f2c33e2..5a33631 100644
--- a/load/load.cxx
+++ b/load/load.cxx
@@ -27,9 +27,6 @@
 
 #include <libbpkg/manifest.hxx>
 
-#include <libbrep/types.hxx>
-#include <libbrep/utility.hxx>
-
 #include <libbrep/package.hxx>
 #include <libbrep/package-odb.hxx>
 #include <libbrep/database-lock.hxx>
@@ -986,7 +983,7 @@ try
 
   if (argc < 2)
   {
-    cerr << "error: configuration file path argument expected" << endl
+    cerr << "error: configuration file expected" << endl
          << help_info << endl;
     return 1;
   }
@@ -1018,7 +1015,7 @@ try
   const string ds ("package");
   if (schema_catalog::current_version (db, ds) != db.schema_version (ds))
   {
-    cerr << "error: database 'package' schema differs from the current one"
+    cerr << "error: package database schema differs from the current one"
          << endl << "  info: use brep-migrate to migrate the database" << endl;
     return 1;
   }
@@ -1091,12 +1088,12 @@ try
 }
 catch (const database_locked&)
 {
-  cerr << "brep-load or brep-migrate instance is running" << endl;
+  cerr << "brep-load or brep-migrate is running" << endl;
   return 2;
 }
 catch (const recoverable& e)
 {
-  cerr << "database recoverable error: " << e << endl;
+  cerr << "recoverable database error: " << e << endl;
   return 3;
 }
 catch (const cli::exception& e)
diff --git a/load/types-parsers.hxx b/load/types-parsers.hxx
index b52c107..74df66e 100644
--- a/load/types-parsers.hxx
+++ b/load/types-parsers.hxx
@@ -5,8 +5,8 @@
 // CLI parsers, included into the generated source files.
 //
 
-#ifndef BREP_LOAD_TYPES_PARSERS_HXX
-#define BREP_LOAD_TYPES_PARSERS_HXX
+#ifndef LOAD_TYPES_PARSERS_HXX
+#define LOAD_TYPES_PARSERS_HXX
 
 #include <libbrep/types.hxx>
 
@@ -25,4 +25,4 @@ namespace cli
   };
 }
 
-#endif // BREP_LOAD_TYPES_PARSERS_HXX
+#endif // LOAD_TYPES_PARSERS_HXX
diff --git a/migrate/.gitignore b/migrate/.gitignore
index 7323078..c27ed44 100644
--- a/migrate/.gitignore
+++ b/migrate/.gitignore
@@ -1,3 +1,2 @@
-*-options
 *-options.?xx
 brep-migrate
diff --git a/migrate/migrate.cli b/migrate/migrate.cli
index 54edd70..1a86626 100644
--- a/migrate/migrate.cli
+++ b/migrate/migrate.cli
@@ -8,7 +8,7 @@ include <cstdint>; // uint16_t
 
 "\section=1"
 "\name=brep-migrate"
-"\summary=create/drop/migrate build2 repository database"
+"\summary=create/drop/migrate brep databases"
 
 {
   "<options>",
@@ -114,12 +114,22 @@ class options
 
 "\h|EXIT STATUS|
 
-\cb{0}      Successful termination.
+\dl|
 
-\cb{1}      Fatal error.
+\li|\cb{0}
 
-\cb{2}      \cb{brep-migrate} or \l{brep-load(1)} instance is running. Try
-            again.
+Success.|
 
-\cb{3}      The database recoverable error. Try again.
+\li|\cb{1}
+
+Fatal error.|
+
+\li|\cb{2}
+
+An instance of \cb{brep-migrate} or \l{brep-load(1)} is already running. Try
+again.|
+
+\li|\cb{3}
+
+Recoverable database error. Try again.||
 "
diff --git a/migrate/migrate.cxx b/migrate/migrate.cxx
index 56e2c83..25b84d0 100644
--- a/migrate/migrate.cxx
+++ b/migrate/migrate.cxx
@@ -15,9 +15,6 @@
 
 #include <libbutl/pager.hxx>
 
-#include <libbrep/types.hxx>
-#include <libbrep/utility.hxx>
-
 #include <libbrep/database-lock.hxx>
 
 #include <migrate/migrate-options.hxx>
@@ -28,7 +25,7 @@ using namespace brep;
 
 // Operation failed, diagnostics has already been issued.
 //
-struct failed: std::exception {};
+struct failed {};
 
 static const char* help_info (
   "  info: run 'brep-migrate --help' for more information");
@@ -349,12 +346,12 @@ try
 }
 catch (const database_locked&)
 {
-  cerr << "brep-migrate or brep-load instance is running" << endl;
+  cerr << "brep-migrate or brep-load is running" << endl;
   return 2;
 }
 catch (const recoverable& e)
 {
-  cerr << "database recoverable error: " << e << endl;
+  cerr << "recoverable database error: " << e << endl;
   return 3;
 }
 catch (const cli::exception& e)
diff --git a/mod/.gitignore b/mod/.gitignore
index ddd62b8..c6e608b 100644
--- a/mod/.gitignore
+++ b/mod/.gitignore
@@ -1,2 +1 @@
-options
 options.?xx