diff options
| author | Karen Arutyunov <karen@codesynthesis.com> | 2020-03-18 22:17:49 +0300 |
|---|---|---|
| committer | Karen Arutyunov <karen@codesynthesis.com> | 2020-03-27 17:28:44 +0300 |
| commit | 35359f038f571dc46de3d14af72a2bc911fb0a24 (patch) | |
| tree | de3e89d678e78b9efc4d395274fd7ccc68f4a213 | |
| parent | 8ad672cc7211952716ffe1fbf76c179b4f1149e3 (diff) | |
Implement brep-monitor
89 files changed, 1800 insertions, 506 deletions
@@ -11,8 +11,9 @@ can be omitted. 1. Create 'brep' User This user will be used to run the brep package database loader, build database -cleaner, and the database schemes migration utility. We will also use its home -directory to build and install the brep module, store its configuration, etc. +cleaner, monitor, and database schemas migration utility. We will also use its +home directory to build and install the brep module, store its configuration, +etc. Note: if the deployment machine employs SELinux, then this approach may require additional configuration steps (not shown) in order to allow Apache2 @@ -194,7 +195,7 @@ CREATE EXTENSION citext; Exit psql (^D) -5. Create Database Schemes and Load Repositories +5. Create Database Schemas and Load Repositories $ mkdir config $ edit config/loadtab # Loader configuration, see brep-load(1). @@ -275,7 +276,7 @@ can also find this fragment in install/share/brep/etc/brep-apache2.conf): # brep-email admin@example.org - # Repository host. It specifies the scheme and the host address (but + # Repository host. It specifies the schema and the host address (but # not the root path; see brep-root below) that will be used whenever # brep needs to construct an absolute URL to one of its locations (for # example, a link to a build log that is being send via email). @@ -384,15 +385,15 @@ $ cd install/share/brep/www/ $ for i in *.scss; do sassc -s compressed $i `basename -s .scss $i`.css; done -8. Setup Periodic Loader and Cleaner Execution +8. Setup Periodic Loader, Cleaner, and Monitor Execution Initially this guide suggested using systemd user session support to run the -loader and the cleaner. However, the current state of user sessions has one -major drawback: they are not started/attached-to when logging in with su -l -(see Debian bug #813789 for details). This limitation makes them unusable in -our setup. If you still would like to use systemd to run the loader and the -cleaner, then you can set it up as a system-wide service which runs the -utilities as the brep user/group. Otherwise, a cron job is a natural choice. +loader, cleaner, and monitor. However, the current state of user sessions has +one major drawback: they are not started/attached-to when logging in with su +-l (see Debian bug #813789 for details). This limitation makes them unusable +in our setup. If you still would like to use systemd to run the utilities, +then you can set it up as a system-wide service which runs them as the brep +user/group. Otherwise, a cron job is a natural choice. Note that the builds cleaner execution is optional and is only required if the build2 build bot functionality is enabled (see the build bot documentation for @@ -402,29 +403,37 @@ parts in the subsequent subsections. If the CI request functionality is enabled you most likely will want to additionally setup the tenants cleanup. +The monitor execution is also optional and currently only makes sense if the +build2 build bot functionality is enabled. Note that you may need to replace +the public toolchain name argument in the monitor utility command with a real +list of toolchain names (and optionally versions) used in the brep build +infrastructure. -8.a Setup Periodic Loader and Cleaner Execution with cron -The following crontab entries will execute the loader every five minutes -and the tenants and builds cleaners once a day at midnight: +8.a Setup Periodic Loader, Cleaner, and Monitor Execution with cron + +The following crontab entries will execute the loader every five minutes, the +tenants and builds cleaners once a day at midnight, and the monitor every hour +(all shifted by a few minutes in order not to clash with other jobs): $ crontab -l MAILTO=<brep-admin-email> PATH=/usr/local/bin:/bin:/usr/bin */5 * * * * $HOME/install/bin/brep-load $HOME/config/loadtab -0 0 * * * $HOME/install/bin/brep-clean tenants 240 -0 0 * * * $HOME/install/bin/brep-clean builds $HOME/config/buildtab +1 0 * * * $HOME/install/bin/brep-clean tenants 240 +2 0 * * * $HOME/install/bin/brep-clean builds $HOME/config/buildtab +3 * * * * $HOME/install/bin/brep-monitor --report-timeout 86400 --clean $HOME/config/brep-module.conf public ^D Note that here we assume that bpkg (which is executed by brep-load) is in one of the PATH's directories (usually /usr/local/bin). -8.b Setup Periodic Loader and Cleaner Execution with systemd +8.b Setup Periodic Loader, Cleaner, and Monitor Execution with systemd In this version we will use the systemd user session to periodically run the -loader and the cleaner as the brep user. If your installation doesn't use -systemd, then a cron job would be a natural alternative (see above). +loader, cleaner, and monitor as the brep user. If your installation doesn't +use systemd, then a cron job would be a natural alternative (see above). As the first step, make sure systemd user sessions support is working for the brep user: @@ -443,6 +452,7 @@ $ sudo loginctl enable-linger brep $ mkdir -p .config/systemd/user $ cp install/share/brep/etc/systemd/brep-load.* .config/systemd/user/ $ cp install/share/brep/etc/systemd/brep-clean.* .config/systemd/user/ +$ cp install/share/brep/etc/systemd/brep-monitor.* .config/systemd/user/ Start the service to make sure there are no issues: @@ -452,16 +462,21 @@ $ journalctl $ systemctl --user start brep-clean.service $ journalctl +$ systemctl --user start brep-monitor.service +$ journalctl + Start the timers and monitor them to make sure they fire: $ systemctl --user start brep-load.timer $ systemctl --user start brep-clean.timer +$ systemctl --user start brep-monitor.timer $ journalctl -f If everything looks good, enable the timer to be started at boot time: $ systemctl --user enable brep-load.timer $ systemctl --user enable brep-clean.timer +$ systemctl --user enable brep-monitor.timer 9. Upgrade Procedure @@ -483,18 +498,20 @@ $ cd brep $ bpkg fetch $ bpkg build brep -If you are using a systemd-based setup, then stop and disable the loader and -the cleaner: +If you are using a systemd-based setup, then stop and disable the loader, +cleaner, and monitor: $ systemctl --user disable --now brep-load.timer $ systemctl --user disable --now brep-clean.timer +$ systemctl --user disable --now brep-monitor.timer $ systemctl --user stop brep-load.service $ systemctl --user stop brep-clean.service +$ systemctl --user stop brep-monitor.service If you are using a cron-based setup, then it is not worth it commenting out the -job entries. If the new version of the loader or the cleaner gets executed -before or during the migration, then it will fail and you will get an email -with the diagnostics. Other than that, it should be harmless. +job entries. If the new version of the brep utilities gets executed before or +during the migration, then it will fail and you will get an email with the +diagnostics. Other than that, it should be harmless. Stop apache: @@ -510,7 +527,7 @@ Review brep-module.conf changes that may need to be merged: $ diff -u install/share/brep/etc/brep-module.conf config/brep-module.conf -Migrate database schemes: +Migrate database schemas: $ install/bin/brep-migrate package $ install/bin/brep-migrate build @@ -521,17 +538,20 @@ is not possible), then one way to do it would be: $ psql -d brep_package -c 'DROP OWNED BY brep' $ psql -d brep_build -c 'DROP OWNED BY brep' -If using systemd, then start and enable the loader and the cleaner: +If using systemd, then start and enable the loader, cleaner, and monitor: $ systemctl --user start brep-load.service $ systemctl --user status brep-load.service $ systemctl --user start brep-clean.service $ systemctl --user status brep-clean.service +$ systemctl --user start brep-monitor.service +$ systemctl --user status brep-monitor.service If everything looks good, enable periodic execution: $ systemctl --user enable --now brep-load.timer $ systemctl --user enable --now brep-clean.timer +$ systemctl --user enable --now brep-monitor.timer If using cron, then simply wait for the next run. diff --git a/INSTALL-DEV b/INSTALL-DEV index 101d9d7..af5c06e 100644 --- a/INSTALL-DEV +++ b/INSTALL-DEV @@ -113,7 +113,7 @@ CREATE EXTENSION citext; Exit psql (^D) -2. Create Database Schemes and Load the Repository +2. Create Database Schemas and Load the Repository All the commands are executed from brep project root. @@ -205,7 +205,7 @@ $ sudo tail -f /var/log/apache2/error.log 4. Reloading During Development -To do a "complete reload" (i.e., recreate database schemes, load the repository +To do a "complete reload" (i.e., recreate database schemas, load the repository data, and reload the Apache2 plugin), execute the following from brep/: $ migrate/brep-migrate --recreate package diff --git a/clean/clean.cli b/clean/clean.cli index 3c710fe..d3be4d6 100644 --- a/clean/clean.cli +++ b/clean/clean.cli @@ -127,8 +127,8 @@ Fatal error.| \li|\cb{2} -An instance of \cb{brep-clean} or \l{brep-migrate(1)} is already running. Try -again.| +An instance of \cb{brep-clean} or some other \cb{brep} utility is already +running. Try again.| \li|\cb{3} diff --git a/clean/clean.cxx b/clean/clean.cxx index d7a7731..5401ab1 100644 --- a/clean/clean.cxx +++ b/clean/clean.cxx @@ -111,8 +111,8 @@ namespace brep ops.db_port (), "options='-c default_transaction_isolation=serializable'"); - // Prevent several brep-clean/migrate instances from updating build - // database simultaneously. + // Prevent several brep utility instances from updating the database + // simultaneously. // database_lock l (db); @@ -316,6 +316,12 @@ namespace brep ? i->second : default_timeout); + // @@ Note that this approach doesn't consider the case when both + // the configuration and the package still exists but the package + // now excludes the configuration (configuration is now of the + // legacy class instead of the default class, etc). We should + // probably re-implement it in a way brep-monitor does it. + // bool cleanup ( // Check that the build is not stale. // diff --git a/doc/buildfile b/doc/buildfile index 2595879..f0a9387 100644 --- a/doc/buildfile +++ b/doc/buildfile @@ -1,10 +1,11 @@ # file : doc/buildfile # license : MIT; see accompanying LICENSE file -cmds = \ -brep-clean \ -brep-load \ -brep-migrate +cmds = \ +brep-clean \ +brep-load \ +brep-migrate \ +brep-monitor ./: {man1 xhtml}{$cmds} \ css{common pre-box man} \ @@ -71,7 +71,7 @@ o="--output-prefix brep-" # #compile "brep" $o --output-prefix "" -pages="clean/clean load/load migrate/migrate" +pages="clean/clean load/load migrate/migrate monitor/monitor" for p in $pages; do compile $p $o diff --git a/doc/manual.cli b/doc/manual.cli index 0b3b038..71a25a5 100644 --- a/doc/manual.cli +++ b/doc/manual.cli @@ -15,7 +15,8 @@ This document describes \c{brep}, the \c{build2} package repository web interface. For the command line interface of \c{brep} utilities refer to the -\l{brep-load(1)}, \l{brep-clean(1)}, and \l{brep-migrate(1)} man pages. +\l{brep-load(1)}, \l{brep-clean(1)}, \l{brep-migrate(1)}, and +\l{brep-monitor(1)} man pages. \h1#submit|Package Submission| diff --git a/etc/brep-module.conf b/etc/brep-module.conf index 458261e..12e96cd 100644 --- a/etc/brep-module.conf +++ b/etc/brep-module.conf @@ -3,6 +3,11 @@ # brep-). See brep(1) for detailed description of each configuration option. # Commented out options indicate their default values. # +# Besides being parsed by the brep module, this file may also be parsed by +# brep utilities that are normally only interested in the subset of the +# options. To simplify skipping of unrecognized, this file must always have an +# option name and its value on the same line. +# # Package search page title. It is placed inside XHTML5 <title> element. # diff --git a/etc/systemd/brep-clean.timer b/etc/systemd/brep-clean.timer index f4c587e..8e1e6e7 100644 --- a/etc/systemd/brep-clean.timer +++ b/etc/systemd/brep-clean.timer @@ -10,9 +10,9 @@ Unit=brep-clean.service # Persistent=false -# Wait 20 seconds until the first run. +# Wait 30 seconds until the first run. # -OnBootSec=20 +OnBootSec=30 # Then wait 5 minutes until the next run. # diff --git a/etc/systemd/brep-monitor.service b/etc/systemd/brep-monitor.service new file mode 100644 index 0000000..0a5c25e --- /dev/null +++ b/etc/systemd/brep-monitor.service @@ -0,0 +1,14 @@ +[Unit] +Description=brep infrastructure monitor service + +[Service] +Type=oneshot +#User=brep +#Group=brep + +# Replace the public toolchain name with a real list of toolchains. +# +ExecStart=/home/brep/install/bin/brep-monitor --report-timeout 86400 --clean /home/brep/config/brep-module.conf public + +[Install] +WantedBy=default.target diff --git a/etc/systemd/brep-monitor.timer b/etc/systemd/brep-monitor.timer new file mode 100644 index 0000000..f5f5a64 --- /dev/null +++ b/etc/systemd/brep-monitor.timer @@ -0,0 +1,23 @@ +[Unit] +Description=brep infrastructure monitor timer +RefuseManualStart=no +RefuseManualStop=no + +[Timer] +Unit=brep-monitor.service + +# Don't keep track of the timer across reboots. +# +Persistent=false + +# Wait 40 seconds until the first run. +# +OnBootSec=40 + +# Then wait 1 hour until the next run. +# +OnUnitInactiveSec=1h + + +[Install] +WantedBy=timers.target diff --git a/libbrep/build-package.hxx b/libbrep/build-package.hxx index 22a8151..702f937 100644 --- a/libbrep/build-package.hxx +++ b/libbrep/build-package.hxx @@ -118,6 +118,8 @@ namespace brep package_id id; upstream_version version; + bool archived; // True if the tenant the package belongs to is archived. + // Database mapping. // #pragma db member(version) set(this.version.init (this.id.version, (?))) diff --git a/libbrep/build.cxx b/libbrep/build.cxx index b6a07c7..db5bda2 100644 --- a/libbrep/build.cxx +++ b/libbrep/build.cxx @@ -80,4 +80,25 @@ namespace brep target (move (trg)) { } + + // build_delay + // + build_delay:: + build_delay (string tnt, + package_name_type pnm, version pvr, + string cfg, + string tnm, version tvr, + timestamp ptm) + : id (package_id (move (tnt), move (pnm), pvr), + move (cfg), + move (tnm), tvr), + tenant (id.package.tenant), + package_name (id.package.name), + package_version (move (pvr)), + configuration (id.configuration), + toolchain_name (id.toolchain_name), + toolchain_version (move (tvr)), + package_timestamp (ptm) + { + } } diff --git a/libbrep/build.hxx b/libbrep/build.hxx index 7e548a4..83b30a8 100644 --- a/libbrep/build.hxx +++ b/libbrep/build.hxx @@ -25,7 +25,7 @@ // #define LIBBREP_BUILD_SCHEMA_VERSION_BASE 9 -#pragma db model version(LIBBREP_BUILD_SCHEMA_VERSION_BASE, 9, closed) +#pragma db model version(LIBBREP_BUILD_SCHEMA_VERSION_BASE, 10, closed) // We have to keep these mappings at the global scope instead of inside // the brep namespace because they need to be also effective in the @@ -212,6 +212,16 @@ namespace brep // optional<result_status> status; + // Time of setting the result status that can be considered as the build + // task completion (currently all the result_status values). Initialized + // with timestamp_nonexistent by default. + // + // Note that in the future we may not consider abort and abnormal as the + // task completion and, for example, proceed with automatic rebuild (the + // flake monitor idea). + // + timestamp_type completion_timestamp; + // May be present only for the building state. // optional<string> agent_fingerprint; @@ -244,6 +254,10 @@ namespace brep // #pragma db member(timestamp) index + // @@ TMP remove when 0.13.0 is released. + // + #pragma db member(completion_timestamp) default(0) + #pragma db member(results) id_column("") value_column("") \ section(results_section) @@ -259,9 +273,7 @@ namespace brep : tenant (id.package.tenant), package_name (id.package.name), configuration (id.configuration), - toolchain_name (id.toolchain_name) - { - } + toolchain_name (id.toolchain_name) {} }; // Note that ADL can't find the equal operator in join conditions, so we use @@ -340,6 +352,68 @@ namespace brep // #pragma db member(result) column("count(" + build::id.package.name + ")") }; + + // Used to track the package build delays since the last build or, if not + // present, since the first opportunity to build the package. + // + #pragma db object pointer(shared_ptr) session + class build_delay + { + public: + using package_name_type = brep::package_name; + + // If toolchain version is empty, then the object represents a minimum + // delay across all versions of the toolchain. + // + build_delay (string tenant, + package_name_type, version, + string configuration, + string toolchain_name, version toolchain_version, + timestamp package_timestamp); + + build_id id; + + string& tenant; // Tracks id.package.tenant. + package_name_type& package_name; // Tracks id.package.name. + upstream_version package_version; // Original of id.package.version. + string& configuration; // Tracks id.configuration. + string& toolchain_name; // Tracks id.toolchain_name. + upstream_version toolchain_version; // Original of id.toolchain_version. + + // Time of the latest delay report. Initialized with timestamp_nonexistent + // by default. + // + timestamp report_timestamp; + + // Time when the package is initially considered as buildable for this + // configuration and toolchain. It is used to track the build delay if the + // build object is absent (the first build task is not yet issued, the + // build is removed by brep-clean, etc). + // + timestamp package_timestamp; + + // Database mapping. + // + #pragma db member(id) id column("") + + #pragma db member(tenant) transient + #pragma db member(package_name) transient + #pragma db member(package_version) \ + set(this.package_version.init (this.id.package.version, (?))) + #pragma db member(configuration) transient + #pragma db member(toolchain_name) transient + #pragma db member(toolchain_version) \ + set(this.toolchain_version.init (this.id.toolchain_version, (?))) + + private: + friend class odb::access; + + build_delay () + : tenant (id.package.tenant), + package_name (id.package.name), + configuration (id.configuration), + toolchain_name (id.toolchain_name) {} + }; } #endif // LIBBREP_BUILD_HXX diff --git a/libbrep/build.xml b/libbrep/build.xml index 3ade7c8..bf8920b 100644 --- a/libbrep/build.xml +++ b/libbrep/build.xml @@ -1,4 +1,44 @@ <changelog xmlns="http://www.codesynthesis.com/xmlns/odb/changelog" database="pgsql" schema-name="build" version="1"> + <changeset version="10"> + <alter-table name="build"> + <add-column name="completion_timestamp" type="BIGINT" null="false" default="0"/> + </alter-table> + <add-table name="build_delay" kind="object"> + <column name="package_tenant" type="TEXT" null="false"/> + <column name="package_name" type="CITEXT" null="false"/> + <column name="package_version_epoch" type="INTEGER" null="false"/> + <column name="package_version_canonical_upstream" type="TEXT" null="false"/> + <column name="package_version_canonical_release" type="TEXT" null="false" options="COLLATE "C""/> + <column name="package_version_revision" type="INTEGER" null="false"/> + <column name="configuration" type="TEXT" null="false"/> + <column name="toolchain_name" type="TEXT" null="false"/> + <column name="toolchain_version_epoch" type="INTEGER" null="false"/> + <column name="toolchain_version_canonical_upstream" type="TEXT" null="false"/> + <column name="toolchain_version_canonical_release" type="TEXT" null="false" options="COLLATE "C""/> + <column name="toolchain_version_revision" type="INTEGER" null="false"/> + <column name="package_version_upstream" type="TEXT" null="false"/> + <column name="package_version_release" type="TEXT" null="true"/> + <column name="toolchain_version_upstream" type="TEXT" null="false"/> + <column name="toolchain_version_release" type="TEXT" null="true"/> + <column name="report_timestamp" type="BIGINT" null="false"/> + <column name="package_timestamp" type="BIGINT" null="false"/> + <primary-key> + <column name="package_tenant"/> + <column name="package_name"/> + <column name="package_version_epoch"/> + <column name="package_version_canonical_upstream"/> + <column name="package_version_canonical_release"/> + <column name="package_version_revision"/> + <column name="configuration"/> + <column name="toolchain_name"/> + <column name="toolchain_version_epoch"/> + <column name="toolchain_version_canonical_upstream"/> + <column name="toolchain_version_canonical_release"/> + <column name="toolchain_version_revision"/> + </primary-key> + </add-table> + </changeset> + <model version="9"> <table name="build" kind="object"> <column name="package_tenant" type="TEXT" null="false"/> diff --git a/libbrep/common.hxx b/libbrep/common.hxx index 44028df..b7fc2da 100644 --- a/libbrep/common.hxx +++ b/libbrep/common.hxx @@ -8,6 +8,8 @@ #include <chrono> #include <type_traits> // static_assert +#include <odb/query.hxx> + #include <libbpkg/package-name.hxx> #include <libbrep/types.hxx> @@ -510,6 +512,34 @@ namespace brep compare_version_ne (x.version, y.version, true); } + // Allow comparing the query members with the query parameters bound by + // reference to variables of the package id type (in particular in the + // prepared queries). + // + // Note that it is not operator==() since the query template parameter type + // can not be deduced from the function parameter types and needs to be + // specified explicitly. + // + template <typename T, typename ID> + inline auto + equal (const ID& x, const package_id& y) + -> decltype (x.tenant == odb::query<T>::_ref (y.tenant) && + x.name == odb::query<T>::_ref (y.name) && + x.version.epoch == odb::query<T>::_ref (y.version.epoch)) + { + using query = odb::query<T>; + + const auto& qv (x.version); + const canonical_version& v (y.version); + + return x.tenant == query::_ref (y.tenant) && + x.name == query::_ref (y.name) && + qv.epoch == query::_ref (v.epoch) && + qv.canonical_upstream == query::_ref (v.canonical_upstream) && + qv.canonical_release == query::_ref (v.canonical_release) && + qv.revision == query::_ref (v.revision); + } + // Repository id comparison operators. // inline bool diff --git a/load/load.cli b/load/load.cli index 1fc2e17..be19ebf 100644 --- a/load/load.cli +++ b/load/load.cli @@ -161,8 +161,8 @@ Fatal error.| \li|\cb{2} -An instance of \cb{brep-load} or \l{brep-migrate(1)} is already running. Try -again.| +An instance of \cb{brep-load} or some other \cb{brep} utility is already +running. Try again.| \li|\cb{3} diff --git a/load/load.cxx b/load/load.cxx index 5638553..bf8584c 100644 --- a/load/load.cxx +++ b/load/load.cxx @@ -1276,7 +1276,7 @@ try ops.db_port (), "options='-c default_transaction_isolation=serializable'"); - // Prevent several brep-load/migrate instances from updating DB + // Prevent several brep utility instances from updating the package database // simultaneously. // database_lock l (db); diff --git a/migrate/migrate.cli b/migrate/migrate.cli index ba9a572..177f991 100644 --- a/migrate/migrate.cli +++ b/migrate/migrate.cli @@ -125,8 +125,8 @@ Fatal error.| \li|\cb{2} -An instance of \cb{brep-migrate} or \l{brep-load(1)} is already running. Try -again.| +An instance of \cb{brep-migrate} or some other \cb{brep} utility is already +running. Try again.| \li|\cb{3} diff --git a/migrate/migrate.cxx b/migrate/migrate.cxx index 8b083fc..81c4543 100644 --- a/migrate/migrate.cxx +++ b/migrate/migrate.cxx @@ -300,12 +300,12 @@ try ops.db_port (), "options='-c default_transaction_isolation=serializable'"); - // Prevent several brep-migrate/load instances from updating DB + // Prevent several brep utility instances from updating the database // simultaneously. // database_lock l (db); - // Currently we don't support data migration for the manual database scheme + // Currently we don't support data migration for the manual database schema // migration. // if (db.schema_migration (db_schema)) diff --git a/mod/.gitignore b/mod/.gitignore index c6e608b..6b64ad0 100644 --- a/mod/.gitignore +++ b/mod/.gitignore @@ -1 +1 @@ -options.?xx +*-options.?xx diff --git a/mod/build-config-module.cxx b/mod/build-config-module.cxx index b1818b7..831cb78 100644 --- a/mod/build-config-module.cxx +++ b/mod/build-config-module.cxx @@ -9,10 +9,9 @@ #include <sstream> #include <libbutl/sha256.mxx> -#include <libbutl/utility.mxx> // throw_generic_error(), alpha(), etc. +#include <libbutl/utility.mxx> // throw_generic_error() #include <libbutl/openssl.mxx> #include <libbutl/filesystem.mxx> // dir_iterator, dir_entry -#include <libbutl/path-pattern.mxx> namespace brep { @@ -157,184 +156,6 @@ namespace brep build_conf_map_ = make_shared<conf_map_type> (move (conf_map)); } - // The default underlying class set expression (see below). - // - static const build_class_expr default_ucs_expr ( - {"default"}, '+', "Default."); - - bool build_config_module:: - exclude (const small_vector<build_class_expr, 1>& exprs, - const vector<build_constraint>& constrs, - const build_config& cfg, - string* reason) const - { - // Save the first sentence of the reason, lower-case the first letter if - // the beginning looks like a word (all subsequent characters until a - // whitespace are lower-case letters). - // - auto sanitize = [] (const string& reason) - { - string r (reason.substr (0, reason.find ('.'))); - - char c (r[0]); // Can be '\0'. - if (alpha (c) && c == ucase (c)) - { - bool word (true); - - for (size_t i (1); - i != r.size () && (c = r[i]) != ' ' && c != '\t' && c != '\n'; - ++i) - { - // Is not a word if contains a non-letter or an upper-case letter. - // - if (!alpha (c) || c == ucase (c)) - { - word = false; - break; - } - } - - if (word) - r[0] = lcase (r[0]); - } - - return r; - }; - - // First, match the configuration against the package underlying build - // class set and expressions. - // - bool m (false); - - // Match the configuration against an expression, updating the match - // result. - // - // We will use a comment of the first encountered excluding expression - // (changing the result from true to false) or non-including one (leaving - // the false result) as an exclusion reason. - // - auto match = [&cfg, &m, reason, &sanitize, this] - (const build_class_expr& e) - { - bool pm (m); - e.match (cfg.classes, build_conf_->class_inheritance_map, m); - - if (reason != nullptr) - { - // Reset the reason which, if saved, makes no sense anymore. - // - if (m) - { - reason->clear (); - } - else if (reason->empty () && - // - // Exclusion. - // - (pm || - // - // Non-inclusion. Make sure that the build class expression - // is empty or starts with an addition (+...). - // - e.expr.empty () || - e.expr.front ().operation == '+')) - { - *reason = sanitize (e.comment); - } - } - }; - - // Determine the underlying class set. Note that in the future we can - // potentially extend the underlying set with special classes. - // - const build_class_expr* ucs ( - !exprs.empty () && !exprs.front ().underlying_classes.empty () - ? &exprs.front () - : nullptr); - - // Note that the combined package build configuration class expression can - // be represented as the underlying class set used as a starting set for - // the original expressions and a restricting set, simultaneously. For - // example, for the expression: - // - // default legacy : -msvc - // - // the resulting expression will be: - // - // +( +default +legacy ) -msvc &( +default +legacy ) - // - // Let's, however, optimize it a bit based on the following facts: - // - // - If the underlying class set expression (+default +legacy in the above - // example) evaluates to false, then the resulting expression also - // evaluates to false due to the trailing '&' operation. Thus, we don't - // need to evaluate further if that's the case. - // - // - On the other hand, if the underlying class set expression evaluates - // to true, then we don't need to apply the trailing '&' operation as it - // cannot affect the result. - // - const build_class_expr& ucs_expr ( - ucs != nullptr - ? build_class_expr (ucs->underlying_classes, '+', ucs->comment) - : default_ucs_expr); - - match (ucs_expr); - - if (m) - { - for (const build_class_expr& e: exprs) - match (e); - } - - // Exclude the configuration if it doesn't match the compound expression. - // - if (!m) - return true; - - // Now check if the configuration is excluded/included via the patterns. - // - // To implement matching of absent name components with wildcard-only - // pattern components we are going to convert names to paths (see - // dash_components_to_path() for details). - // - // And if any of the build-{include,exclude} values (which is legal) or - // the build configuration name/target (illegal) are invalid paths, then - // we assume no match. - // - if (!constrs.empty ()) - try - { - path cn (dash_components_to_path (cfg.name)); - path tg (dash_components_to_path (cfg.target.string ())); - - for (const build_constraint& c: constrs) - { - if (path_match (cn, - dash_components_to_path (c.config), - dir_path () /* start */, - path_match_flags::match_absent) && - (!c.target || - path_match (tg, - dash_components_to_path (*c.target), - dir_path () /* start */, - path_match_flags::match_absent))) - { - if (!c.exclusion) - return false; - - if (reason != nullptr) - *reason = sanitize (c.comment); - - return true; - } - } - } - catch (const invalid_path&) {} - - return false; - } - bool build_config_module:: belongs (const bbot::build_config& cfg, const char* cls) const { @@ -360,59 +181,4 @@ namespace brep return false; } - - path build_config_module:: - dash_components_to_path (const string& pattern) - { - string r; - size_t nstar (0); - for (const path_pattern_term& pt: path_pattern_iterator (pattern)) - { - switch (pt.type) - { - case path_pattern_term_type::star: - { - // Replace ** with */**/* and skip all the remaining stars that may - // follow in this sequence. - // - if (nstar == 0) - r += "*"; - else if (nstar == 1) - r += "/**/*"; // The first star is already copied. - - break; - } - case path_pattern_term_type::literal: - { - // Replace '-' with '/' and fall through otherwise. - // - if (get_literal (pt) == '-') - { - r += '/'; - break; - } - } - // Fall through. - default: - { - r.append (pt.begin, pt.end); // Copy the pattern term as is. - } - } - - nstar = pt.star () ? nstar + 1 : 0; - } - - // Append the trailing slash to match the resulting paths as directories. - // This is required for the trailing /* we could append to match absent - // directory path components (see path_match_flags::match_absent for - // details). - // - // Note that valid dash components may not contain a trailing dash. - // Anyway, any extra trailing slashes will be ignored by the path - // constructor. - // - r += '/'; - - return path (move (r)); - } } diff --git a/mod/build-config-module.hxx b/mod/build-config-module.hxx index 4b23056..04fd5b1 100644 --- a/mod/build-config-module.hxx +++ b/mod/build-config-module.hxx @@ -5,7 +5,6 @@ #define MOD_BUILD_CONFIG_MODULE_HXX #include <map> -#include <algorithm> // find() #include <libbutl/utility.mxx> // compare_c_string @@ -16,8 +15,8 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/build-config.hxx> +#include <mod/module-options.hxx> // Base class for modules that utilize the build controller configuration. // @@ -39,17 +38,18 @@ namespace brep void init (const options::build&); - // Return true if the specified build configuration is excluded by a - // package based on its underlying build class set, build class - // expressions, and build constraints, potentially extending the - // underlying set with the special classes. Set the exclusion reason if - // requested. - // bool - exclude (const small_vector<bpkg::build_class_expr, 1>&, - const vector<bpkg::build_constraint>&, - const bbot::build_config&, - string* reason = nullptr) const; + exclude (const small_vector<bpkg::build_class_expr, 1>& exprs, + const vector<bpkg::build_constraint>& constrs, + const bbot::build_config& cfg, + string* reason = nullptr) const + { + return brep::exclude (exprs, + constrs, + cfg, + build_conf_->class_inheritance_map, + reason); + } // Check if the configuration belongs to the specified class. // @@ -62,20 +62,6 @@ namespace brep return belongs (cfg, cls.c_str ()); } - // Convert dash-separated components (target, build configuration name, - // machine name) or a pattern thereof into a path, replacing dashes with - // slashes (directory separators), `**` with `*/**/*`, and appending the - // trailing slash for a subsequent match using the path_match() - // functionality (the idea here is for `linux**` to match `linux-gcc` - // which is quite natural to expect). Throw invalid_path if the resulting - // path is invalid. - // - // Note that the match_absent path match flag must be used for the above - // `**` transformation to work. - // - static path - dash_components_to_path (const string&); - // Configuration/toolchain combination that, in particular, can be used as // a set value. // diff --git a/mod/build-config.cxx b/mod/build-config.cxx new file mode 100644 index 0000000..2d64aec --- /dev/null +++ b/mod/build-config.cxx @@ -0,0 +1,249 @@ +// file : mod/build-config-module.cxx -*- C++ -*- +// license : MIT; see accompanying LICENSE file + +#include <mod/build-config.hxx> + +#include <libbutl/utility.mxx> // alpha(), etc. +#include <libbutl/path-pattern.mxx> + +namespace brep +{ + using namespace std; + using namespace butl; + using namespace bpkg; + using namespace bbot; + + // The default underlying class set expression (see below). + // + static const build_class_expr default_ucs_expr ( + {"default"}, '+', "Default."); + + bool + exclude (const small_vector<build_class_expr, 1>& exprs, + const vector<build_constraint>& constrs, + const build_config& cfg, + const map<string, string>& class_inheritance_map, + string* reason) + { + // Save the first sentence of the reason, lower-case the first letter if + // the beginning looks like a word (all subsequent characters until a + // whitespace are lower-case letters). + // + auto sanitize = [] (const string& reason) + { + string r (reason.substr (0, reason.find ('.'))); + + char c (r[0]); // Can be '\0'. + if (alpha (c) && c == ucase (c)) + { + bool word (true); + + for (size_t i (1); + i != r.size () && (c = r[i]) != ' ' && c != '\t' && c != '\n'; + ++i) + { + // Is not a word if contains a non-letter or an upper-case letter. + // + if (!alpha (c) || c == ucase (c)) + { + word = false; + break; + } + } + + if (word) + r[0] = lcase (r[0]); + } + + return r; + }; + + // First, match the configuration against the package underlying build + // class set and expressions. + // + bool m (false); + + // Match the configuration against an expression, updating the match + // result. + // + // We will use a comment of the first encountered excluding expression + // (changing the result from true to false) or non-including one (leaving + // the false result) as an exclusion reason. + // + auto match = [&cfg, &m, reason, &sanitize, &class_inheritance_map] + (const build_class_expr& e) + { + bool pm (m); + e.match (cfg.classes, class_inheritance_map, m); + + if (reason != nullptr) + { + // Reset the reason which, if saved, makes no sense anymore. + // + if (m) + { + reason->clear (); + } + else if (reason->empty () && + // + // Exclusion. + // + (pm || + // + // Non-inclusion. Make sure that the build class expression + // is empty or starts with an addition (+...). + // + e.expr.empty () || + e.expr.front ().operation == '+')) + { + *reason = sanitize (e.comment); + } + } + }; + + // Determine the underlying class set. Note that in the future we can + // potentially extend the underlying set with special classes. + // + const build_class_expr* ucs ( + !exprs.empty () && !exprs.front ().underlying_classes.empty () + ? &exprs.front () + : nullptr); + + // Note that the combined package build configuration class expression can + // be represented as the underlying class set used as a starting set for + // the original expressions and a restricting set, simultaneously. For + // example, for the expression: + // + // default legacy : -msvc + // + // the resulting expression will be: + // + // +( +default +legacy ) -msvc &( +default +legacy ) + // + // Let's, however, optimize it a bit based on the following facts: + // + // - If the underlying class set expression (+default +legacy in the above + // example) evaluates to false, then the resulting expression also + // evaluates to false due to the trailing '&' operation. Thus, we don't + // need to evaluate further if that's the case. + // + // - On the other hand, if the underlying class set expression evaluates + // to true, then we don't need to apply the trailing '&' operation as it + // cannot affect the result. + // + const build_class_expr& ucs_expr ( + ucs != nullptr + ? build_class_expr (ucs->underlying_classes, '+', ucs->comment) + : default_ucs_expr); + + match (ucs_expr); + + if (m) + { + for (const build_class_expr& e: exprs) + match (e); + } + + // Exclude the configuration if it doesn't match the compound expression. + // + if (!m) + return true; + + // Now check if the configuration is excluded/included via the patterns. + // + // To implement matching of absent name components with wildcard-only + // pattern components we are going to convert names to paths (see + // dash_components_to_path() for details). + // + // And if any of the build-{include,exclude} values (which is legal) or + // the build configuration name/target (illegal) are invalid paths, then + // we assume no match. + // + if (!constrs.empty ()) + try + { + path cn (dash_components_to_path (cfg.name)); + path tg (dash_components_to_path (cfg.target.string ())); + + for (const build_constraint& c: constrs) + { + if (path_match (cn, + dash_components_to_path (c.config), + dir_path () /* start */, + path_match_flags::match_absent) && + (!c.target || + path_match (tg, + dash_components_to_path (*c.target), + dir_path () /* start */, + path_match_flags::match_absent))) + { + if (!c.exclusion) + return false; + + if (reason != nullptr) + *reason = sanitize (c.comment); + + return true; + } + } + } + catch (const invalid_path&) {} + + return false; + } + + path + dash_components_to_path (const string& pattern) + { + string r; + size_t nstar (0); + for (const path_pattern_term& pt: path_pattern_iterator (pattern)) + { + switch (pt.type) + { + case path_pattern_term_type::star: + { + // Replace ** with */**/* and skip all the remaining stars that may + // follow in this sequence. + // + if (nstar == 0) + r += "*"; + else if (nstar == 1) + r += "/**/*"; // The first star is already copied. + + break; + } + case path_pattern_term_type::literal: + { + // Replace '-' with '/' and fall through otherwise. + // + if (get_literal (pt) == '-') + { + r += '/'; + break; + } + } + // Fall through. + default: + { + r.append (pt.begin, pt.end); // Copy the pattern term as is. + } + } + + nstar = pt.star () ? nstar + 1 : 0; + } + + // Append the trailing slash to match the resulting paths as directories. + // This is required for the trailing /* we could append to match absent + // directory path components (see path_match_flags::match_absent for + // details). + // + // Note that valid dash components may not contain a trailing dash. + // Anyway, any extra trailing slashes will be ignored by the path + // constructor. + // + r += '/'; + + return path (move (r)); + } +} diff --git a/mod/build-config.hxx b/mod/build-config.hxx new file mode 100644 index 0000000..d5e44ce --- /dev/null +++ b/mod/build-config.hxx @@ -0,0 +1,45 @@ +// file : mod/build-config.hxx -*- C++ -*- +// license : MIT; see accompanying LICENSE file + +#ifndef MOD_BUILD_CONFIG_HXX +#define MOD_BUILD_CONFIG_HXX + +#include <map> + +#include <libbpkg/manifest.hxx> + +#include <libbbot/build-config.hxx> + +#include <libbrep/types.hxx> +#include <libbrep/utility.hxx> + +namespace brep +{ + // Return true if the specified build configuration is excluded by a package + // based on its underlying build class set, build class expressions, and + // build constraints, potentially extending the underlying set with the + // special classes. Set the exclusion reason if requested. + // + bool + exclude (const small_vector<bpkg::build_class_expr, 1>&, + const vector<bpkg::build_constraint>&, + const bbot::build_config&, + const std::map<string, string>& class_inheritance_map, + string* reason = nullptr); + + // Convert dash-separated components (target, build configuration name, + // machine name) or a pattern thereof into a path, replacing dashes with + // slashes (directory separators), `**` with `*/**/*`, and appending the + // trailing slash for a subsequent match using the path_match() + // functionality (the idea here is for `linux**` to match `linux-gcc` which + // is quite natural to expect). Throw invalid_path if the resulting path is + // invalid. + // + // Note that the match_absent path match flag must be used for the above + // `**` transformation to work. + // + path + dash_components_to_path (const string&); +} + +#endif // MOD_BUILD_CONFIG diff --git a/mod/build.cxx b/mod/build.cxx index cdbaa60..5b9d8aa 100644 --- a/mod/build.cxx +++ b/mod/build.cxx @@ -3,7 +3,7 @@ #include <mod/build.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/mime-url-encoding.hxx> #include <mod/utility.hxx> diff --git a/mod/buildfile b/mod/buildfile index 9300faf..ca46bc4 100644 --- a/mod/buildfile +++ b/mod/buildfile @@ -19,25 +19,38 @@ import libs += libbpkg%lib{bpkg} import libs += libbbot%lib{bbot} include ../libbrep/ -include ../web/ -mod{brep}: {hxx ixx txx cxx}{* -options} \ - {hxx ixx cxx}{ options} \ - ../libbrep/lib{brep} ../web/libus{web} $libs +include ../web/xhtml/ +include ../web/server/ + +./: mod{brep} {libue libus}{mod} + +libu_src = options-types types-parsers build-config + +mod{brep}: {hxx ixx txx cxx}{* -module-options -{$libu_src}} \ + libus{mod} ../libbrep/lib{brep} ../web/server/libus{web-server} \ + $libs + +{libue libus}{mod}: {hxx ixx cxx}{module-options} \ + {hxx ixx txx cxx}{+{$libu_src} } \ + $libs + +libus{mod}: ../web/xhtml/libus{xhtml} +libue{mod}: ../web/xhtml/libue{xhtml} # Generated options parser. # if $cli.configured { - cli.cxx{options}: cli{options} + cli.cxx{module-options}: cli{module} # Set option prefix to the empty value to handle all unknown request # parameters uniformly with a single catch block. # - cli.options += --std c++11 -I $src_root --include-with-brackets \ ---include-prefix mod --guard-prefix MOD --generate-specifier \ ---cxx-prologue "#include <mod/types-parsers.hxx>" \ ---cli-namespace brep::cli --generate-file-scanner --suppress-usage \ + cli.options += --std c++11 -I $src_root --include-with-brackets \ +--include-prefix mod --guard-prefix MOD --generate-specifier \ +--cxx-prologue "#include <mod/types-parsers.hxx>" \ +--cli-namespace brep::cli --generate-file-scanner --option-length 38 \ --generate-modifier --generate-description --option-prefix "" # Include the generated cli files into the distribution and don't remove diff --git a/mod/database-module.cxx b/mod/database-module.cxx index 5516730..f598bfd 100644 --- a/mod/database-module.cxx +++ b/mod/database-module.cxx @@ -5,8 +5,8 @@ #include <odb/exceptions.hxx> -#include <mod/options.hxx> #include <mod/database.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/database-module.hxx b/mod/database-module.hxx index a41752d..f72ba83 100644 --- a/mod/database-module.hxx +++ b/mod/database-module.hxx @@ -10,7 +10,7 @@ #include <libbrep/utility.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/mod-build-configs.cxx b/mod/mod-build-configs.cxx index 8efc6c9..6731b28 100644 --- a/mod/mod-build-configs.cxx +++ b/mod/mod-build-configs.cxx @@ -7,11 +7,12 @@ #include <libstudxml/serializer.hxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> + +#include <web/xhtml/serialization.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace bbot; diff --git a/mod/mod-build-configs.hxx b/mod/mod-build-configs.hxx index 333680a..562ac6d 100644 --- a/mod/mod-build-configs.hxx +++ b/mod/mod-build-configs.hxx @@ -8,7 +8,7 @@ #include <libbrep/utility.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/build-config-module.hxx> namespace brep diff --git a/mod/mod-build-force.cxx b/mod/mod-build-force.cxx index 4dc71c8..bd172e3 100644 --- a/mod/mod-build-force.cxx +++ b/mod/mod-build-force.cxx @@ -8,12 +8,12 @@ #include <odb/database.hxx> #include <odb/transaction.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace bbot; diff --git a/mod/mod-build-force.hxx b/mod/mod-build-force.hxx index 7b6b3b6..22df383 100644 --- a/mod/mod-build-force.hxx +++ b/mod/mod-build-force.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-build-log.cxx b/mod/mod-build-log.cxx index 16cc965..3032e52 100644 --- a/mod/mod-build-log.cxx +++ b/mod/mod-build-log.cxx @@ -10,12 +10,12 @@ #include <libbutl/timestamp.mxx> // to_stream() -#include <web/module.hxx> +#include <web/server/module.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace bbot; diff --git a/mod/mod-build-log.hxx b/mod/mod-build-log.hxx index 9f9d1d9..a2f4e48 100644 --- a/mod/mod-build-log.hxx +++ b/mod/mod-build-log.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-build-result.cxx b/mod/mod-build-result.cxx index b3467d2..734ea5c 100644 --- a/mod/mod-build-result.cxx +++ b/mod/mod-build-result.cxx @@ -15,15 +15,15 @@ #include <libbbot/manifest.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> #include <libbrep/package.hxx> #include <libbrep/package-odb.hxx> -#include <mod/build.hxx> // *_url() -#include <mod/options.hxx> +#include <mod/build.hxx> // *_url() +#include <mod/module-options.hxx> using namespace std; using namespace butl; @@ -409,6 +409,7 @@ handle (request& rq, response&) b->results = move (rqm.result.results); b->timestamp = system_clock::now (); + b->completion_timestamp = b->timestamp; build_db_->update (b); diff --git a/mod/mod-build-result.hxx b/mod/mod-build-result.hxx index b3911e1..71a60f9 100644 --- a/mod/mod-build-result.hxx +++ b/mod/mod-build-result.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-build-task.cxx b/mod/mod-build-task.cxx index c232815..17bc15e 100644 --- a/mod/mod-build-task.cxx +++ b/mod/mod-build-task.cxx @@ -22,14 +22,14 @@ #include <libbbot/manifest.hxx> #include <libbbot/build-config.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> #include <libbrep/build-package.hxx> #include <libbrep/build-package-odb.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace butl; @@ -384,28 +384,18 @@ handle (request& rq, response& rs) using prep_bld_query = prepared_query<build>; package_id id; - const auto& qv (bld_query::id.package.version); bld_query bq ( - bld_query::id.package.tenant == bld_query::_ref (id.tenant) && - - bld_query::id.package.name == bld_query::_ref (id.name) && - - qv.epoch == bld_query::_ref (id.version.epoch) && - qv.canonical_upstream == - bld_query::_ref (id.version.canonical_upstream) && - qv.canonical_release == - bld_query::_ref (id.version.canonical_release) && - qv.revision == bld_query::_ref (id.version.revision) && + equal<build> (bld_query::id.package, id) && bld_query::id.configuration.in_range (cfg_names.begin (), - cfg_names.end ()) && + cfg_names.end ()) && - bld_query::id.toolchain_name == tqm.toolchain_name && + bld_query::id.toolchain_name == tqm.toolchain_name && compare_version_eq (bld_query::id.toolchain_version, canonical_version (toolchain_version), - true /* revision */) && + true /* revision */) && (bld_query::state == "built" || ((bld_query::force == "forcing" && diff --git a/mod/mod-build-task.hxx b/mod/mod-build-task.hxx index 5f4c14a..7875db1 100644 --- a/mod/mod-build-task.hxx +++ b/mod/mod-build-task.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-builds.cxx b/mod/mod-builds.cxx index 77ebc05..ab9e93e 100644 --- a/mod/mod-builds.cxx +++ b/mod/mod-builds.cxx @@ -16,9 +16,10 @@ #include <libbbot/manifest.hxx> // to_result_status(), to_string(result_status) -#include <web/xhtml.hxx> -#include <web/module.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/module.hxx> +#include <web/server/mime-url-encoding.hxx> + +#include <web/xhtml/serialization.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> @@ -26,7 +27,7 @@ #include <libbrep/build-package-odb.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace butl; @@ -231,7 +232,10 @@ build_query (const brep::cstrings* configs, else { query sq (qb::status == rs); - result_status st (to_result_status(rs)); // May throw invalid_argument. + + // May throw invalid_argument. + // + result_status st (to_result_status (rs)); if (st != result_status::success) { @@ -312,22 +316,6 @@ package_query (const brep::params::builds& params, return q; } -template <typename T, typename ID> -static inline query<T> -package_id_eq (const ID& x, const brep::package_id& y) -{ - using query = query<T>; - const auto& qv (x.version); - - return - x.tenant == query::_ref (y.tenant) && - x.name == query::_ref (y.name) && - qv.epoch == query::_ref (y.version.epoch) && - qv.canonical_upstream == query::_ref (y.version.canonical_upstream) && - qv.canonical_release == query::_ref (y.version.canonical_release) && - qv.revision == query::_ref (y.version.revision); -} - static const vector<pair<string, string>> build_results ({ {"unbuilt", "<unbuilt>"}, {"*", "*"}, @@ -821,9 +809,8 @@ handle (request& rq, response& rs) const auto& bid (bld_query::build::id); - bld_query bq ( - package_id_eq<package_build_count> (bid.package, id) && - bid.configuration == bld_query::_ref (config) && + bld_query bq (equal<package_build_count> (bid.package, id) && + bid.configuration == bld_query::_ref (config) && // Note that the query already constrains configurations via the // configuration name and the tenant via the build package id. @@ -936,7 +923,7 @@ handle (request& rq, response& rs) package_id id; bld_query bq ( - package_id_eq<package_build> (bld_query::build::id.package, id) && + equal<package_build> (bld_query::build::id.package, id) && // Note that the query already constrains the tenant via the build // package id. diff --git a/mod/mod-builds.hxx b/mod/mod-builds.hxx index 714b374..0aa7916 100644 --- a/mod/mod-builds.hxx +++ b/mod/mod-builds.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-ci.cxx b/mod/mod-ci.cxx index 77377eb..d2da93f 100644 --- a/mod/mod-ci.cxx +++ b/mod/mod-ci.cxx @@ -17,11 +17,12 @@ #include <libbpkg/manifest.hxx> #include <libbpkg/package-name.hxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> + +#include <web/xhtml/serialization.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/external-handler.hxx> using namespace std; @@ -116,8 +117,8 @@ handle (request& rq, response& rs) // latter case we will always respond with the same neutral message for // security reason, logging the error details. Note that descriptions of // exceptions caught by the web server are returned to the client (see - // web/module.hxx for details), and we want to avoid this when there is a - // danger of exposing sensitive data. + // web/server/module.hxx for details), and we want to avoid this when there + // is a danger of exposing sensitive data. // // Also we will pass through exceptions thrown by the underlying API, unless // we need to handle them or add details for the description, in which case diff --git a/mod/mod-ci.hxx b/mod/mod-ci.hxx index 1228714..431f53b 100644 --- a/mod/mod-ci.hxx +++ b/mod/mod-ci.hxx @@ -4,13 +4,13 @@ #ifndef MOD_MOD_CI_HXX #define MOD_MOD_CI_HXX -#include <web/xhtml-fragment.hxx> +#include <web/xhtml/fragment.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/mod-package-details.cxx b/mod/mod-package-details.cxx index c7973d3..e0bd1ef 100644 --- a/mod/mod-package-details.cxx +++ b/mod/mod-package-details.cxx @@ -9,15 +9,16 @@ #include <odb/database.hxx> #include <odb/transaction.hxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/module.hxx> +#include <web/server/mime-url-encoding.hxx> + +#include <web/xhtml/serialization.hxx> #include <libbrep/package.hxx> #include <libbrep/package-odb.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace odb::core; using namespace brep::cli; diff --git a/mod/mod-package-details.hxx b/mod/mod-package-details.hxx index 16f8c3e..e1b0a9c 100644 --- a/mod/mod-package-details.hxx +++ b/mod/mod-package-details.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> namespace brep diff --git a/mod/mod-package-version-details.cxx b/mod/mod-package-version-details.cxx index cde65b0..bfc08b0 100644 --- a/mod/mod-package-version-details.cxx +++ b/mod/mod-package-version-details.cxx @@ -9,9 +9,10 @@ #include <odb/database.hxx> #include <odb/transaction.hxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/module.hxx> +#include <web/server/mime-url-encoding.hxx> + +#include <web/xhtml/serialization.hxx> #include <libbrep/build.hxx> #include <libbrep/build-odb.hxx> @@ -19,7 +20,7 @@ #include <libbrep/package-odb.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace butl; diff --git a/mod/mod-package-version-details.hxx b/mod/mod-package-version-details.hxx index 8d0d373..a88d6c2 100644 --- a/mod/mod-package-version-details.hxx +++ b/mod/mod-package-version-details.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> #include <mod/build-config-module.hxx> diff --git a/mod/mod-packages.cxx b/mod/mod-packages.cxx index 81cf83c..65c7c5b 100644 --- a/mod/mod-packages.cxx +++ b/mod/mod-packages.cxx @@ -10,15 +10,16 @@ #include <odb/transaction.hxx> #include <odb/schema-catalog.hxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/module.hxx> +#include <web/server/mime-url-encoding.hxx> + +#include <web/xhtml/serialization.hxx> #include <libbrep/package.hxx> #include <libbrep/package-odb.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace odb::core; using namespace brep::cli; diff --git a/mod/mod-packages.hxx b/mod/mod-packages.hxx index d1c4677..611d63c 100644 --- a/mod/mod-packages.hxx +++ b/mod/mod-packages.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> namespace brep diff --git a/mod/mod-repository-details.cxx b/mod/mod-repository-details.cxx index 988c445..813b738 100644 --- a/mod/mod-repository-details.cxx +++ b/mod/mod-repository-details.cxx @@ -12,15 +12,16 @@ #include <libbutl/timestamp.mxx> // to_string() -#include <web/xhtml.hxx> -#include <web/module.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/module.hxx> +#include <web/server/mime-url-encoding.hxx> + +#include <web/xhtml/serialization.hxx> #include <libbrep/package.hxx> #include <libbrep/package-odb.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace odb::core; diff --git a/mod/mod-repository-details.hxx b/mod/mod-repository-details.hxx index bd4b3ba..e83831d 100644 --- a/mod/mod-repository-details.hxx +++ b/mod/mod-repository-details.hxx @@ -7,7 +7,7 @@ #include <libbrep/types.hxx> #include <libbrep/utility.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/database-module.hxx> namespace brep diff --git a/mod/mod-repository-root.cxx b/mod/mod-repository-root.cxx index b6c54b8..02d6c93 100644 --- a/mod/mod-repository-root.cxx +++ b/mod/mod-repository-root.cxx @@ -10,10 +10,10 @@ #include <sstream> #include <algorithm> // find() -#include <web/module.hxx> +#include <web/server/module.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/mod-ci.hxx> #include <mod/mod-submit.hxx> diff --git a/mod/mod-repository-root.hxx b/mod/mod-repository-root.hxx index ac4b254..9e28797 100644 --- a/mod/mod-repository-root.hxx +++ b/mod/mod-repository-root.hxx @@ -8,7 +8,7 @@ #include <libbrep/utility.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/mod-submit.cxx b/mod/mod-submit.cxx index 0dea2b7..9c93a36 100644 --- a/mod/mod-submit.cxx +++ b/mod/mod-submit.cxx @@ -14,11 +14,12 @@ #include <libbutl/manifest-types.mxx> #include <libbutl/manifest-serializer.mxx> -#include <web/xhtml.hxx> -#include <web/module.hxx> +#include <web/server/module.hxx> + +#include <web/xhtml/serialization.hxx> #include <mod/page.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> #include <mod/external-handler.hxx> using namespace std; diff --git a/mod/mod-submit.hxx b/mod/mod-submit.hxx index 96a60f9..fc5f8d4 100644 --- a/mod/mod-submit.hxx +++ b/mod/mod-submit.hxx @@ -4,13 +4,13 @@ #ifndef MOD_MOD_SUBMIT_HXX #define MOD_MOD_SUBMIT_HXX -#include <web/xhtml-fragment.hxx> +#include <web/xhtml/fragment.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> #include <mod/module.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/options.cli b/mod/module.cli index f02d7a6..fa1d2cc 100644 --- a/mod/options.cli +++ b/mod/module.cli @@ -3,7 +3,7 @@ include <libbpkg/manifest.hxx>; // repository_location -include <web/xhtml-fragment.hxx>; +include <web/xhtml/fragment.hxx>; include <libbrep/types.hxx>; @@ -37,7 +37,7 @@ namespace brep dir_path root = "/" { - "<path>" + "<path>", "Repository root. That is, this is the part of the URL between the host name and the start of the repository. For example, root value '\cb{/pkg}' means the repository URL is \cb{http://example.org/pkg/}. @@ -214,7 +214,7 @@ namespace brep { "<user>", "Build database execution user name. If not empty then the login - user will be switched (with \c{SET ROLE}) to this user prior to + user will be switched (with \c{SET ROLE}) to this user prior to executing any statements. If not specified, then \cb{brep} is used." } @@ -565,7 +565,7 @@ namespace brep string root-tenant-view = "packages" { - "<service>" + "<service>", "The default view to display for the tenant repository root. The <service> argument is one of the supported services (\c{packages}, \c{builds}, \c{submit}, \c{ci}, etc). The default service is diff --git a/mod/module.cxx b/mod/module.cxx index 8f306fd..06799d7 100644 --- a/mod/module.cxx +++ b/mod/module.cxx @@ -10,10 +10,10 @@ #include <cstring> // strchr() #include <functional> // bind() -#include <web/module.hxx> -#include <web/apache/log.hxx> +#include <web/server/module.hxx> +#include <web/server/apache/log.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace placeholders; // For std::bind's _1, etc. diff --git a/mod/module.hxx b/mod/module.hxx index 2c62166..b3ed67b 100644 --- a/mod/module.hxx +++ b/mod/module.hxx @@ -4,14 +4,14 @@ #ifndef MOD_MODULE_HXX #define MOD_MODULE_HXX -#include <web/module.hxx> +#include <web/server/module.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> #include <mod/utility.hxx> -#include <mod/options.hxx> #include <mod/diagnostics.hxx> +#include <mod/module-options.hxx> namespace brep { diff --git a/mod/page.cxx b/mod/page.cxx index 64e31c0..c7dc403 100644 --- a/mod/page.cxx +++ b/mod/page.cxx @@ -16,9 +16,10 @@ #include <libbutl/url.mxx> -#include <web/xhtml.hxx> -#include <web/xhtml-fragment.hxx> -#include <web/mime-url-encoding.hxx> +#include <web/xhtml/fragment.hxx> +#include <web/xhtml/serialization.hxx> + +#include <web/server/mime-url-encoding.hxx> #include <libbrep/package.hxx> #include <libbrep/package-odb.hxx> diff --git a/mod/page.hxx b/mod/page.hxx index 8c92d10..49d8608 100644 --- a/mod/page.hxx +++ b/mod/page.hxx @@ -8,7 +8,7 @@ #include <libbbot/manifest.hxx> -#include <web/xhtml-fragment.hxx> +#include <web/xhtml/fragment.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> diff --git a/mod/services.cxx b/mod/services.cxx index 7739011..b17e32e 100644 --- a/mod/services.cxx +++ b/mod/services.cxx @@ -3,7 +3,7 @@ #include <ap_config.h> // AP_MODULE_DECLARE_DATA -#include <web/apache/service.hxx> +#include <web/server/apache/service.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> diff --git a/mod/types-parsers.cxx b/mod/types-parsers.cxx index 70d77dd..ceaab29 100644 --- a/mod/types-parsers.cxx +++ b/mod/types-parsers.cxx @@ -3,7 +3,7 @@ #include <mod/types-parsers.hxx> -#include <mod/options.hxx> +#include <mod/module-options.hxx> using namespace std; using namespace bpkg; diff --git a/mod/types-parsers.hxx b/mod/types-parsers.hxx index a81ef90..091c868 100644 --- a/mod/types-parsers.hxx +++ b/mod/types-parsers.hxx @@ -9,7 +9,7 @@ #include <libbpkg/manifest.hxx> // repository_location -#include <web/xhtml-fragment.hxx> +#include <web/xhtml/fragment.hxx> #include <libbrep/types.hxx> #include <libbrep/utility.hxx> diff --git a/monitor/.gitignore b/monitor/.gitignore new file mode 100644 index 0000000..21c0e0b --- /dev/null +++ b/monitor/.gitignore @@ -0,0 +1,2 @@ +*-options.?xx +brep-monitor diff --git a/monitor/buildfile b/monitor/buildfile new file mode 100644 index 0000000..dc49a98 --- /dev/null +++ b/monitor/buildfile @@ -0,0 +1,45 @@ +# file : monitor/buildfile +# license : MIT; see accompanying LICENSE file + +import libs = libodb%lib{odb} +import libs += libodb-pgsql%lib{odb-pgsql} +import libs += libbutl%lib{butl} +import libs += libbbot%lib{bbot} + +include ../libbrep/ +include ../mod/ + +exe{brep-monitor}: {hxx ixx cxx}{* -*-options} \ + {hxx ixx cxx}{monitor-options module-options} \ + ../mod/libue{mod} ../libbrep/lib{brep} $libs + +# Build options. +# +obj{monitor}: cxx.poptions += -DBREP_COPYRIGHT=\"$copyright\" + +# Generated options parser. +# +if $cli.configured +{ + cli.cxx{monitor-options}: cli{monitor} + cli.cxx{module-options}: cli{module} + + cli.options += --std c++11 -I $src_root --include-with-brackets \ +--include-prefix monitor --guard-prefix MONITOR --generate-specifier \ +--cli-namespace brep::cli + + cli.cxx{monitor-options}: cli.options += \ +--page-usage print_ --ansi-color --long-usage + + cli.cxx{module-options}: cli.options += --suppress-usage --generate-parse + + # Include the generated cli files into the distribution and don't remove + # them when cleaning in src (so that clean results in a state identical to + # distributed). + # + cli.cxx{*}: + { + dist = true + clean = ($src_root != $out_root) + } +} diff --git a/monitor/module.cli b/monitor/module.cli new file mode 100644 index 0000000..c299c5f --- /dev/null +++ b/monitor/module.cli @@ -0,0 +1,16 @@ +// file : monitor/module.cli +// license : MIT; see accompanying LICENSE file + +include <mod/module.cli>; + +namespace brep +{ + namespace options + { + // brep web module configuration options we are interested in. + // + class module: build_task + { + }; + } +} diff --git a/monitor/monitor.cli b/monitor/monitor.cli new file mode 100644 index 0000000..33b05f7 --- /dev/null +++ b/monitor/monitor.cli @@ -0,0 +1,176 @@ +// file : monitor/monitor.cli +// license : MIT; see accompanying LICENSE file + +include <vector>; +include <string>; +include <cstddef>; // size_t +include <cstdint>; // uint16_t + +include <mod/module.cli>; // Reuse CLI support types. + +"\section=1" +"\name=brep-monitor" +"\summary=monitor brep infrastructure" + +namespace brep +{ + namespace options + { + { + "<options> <brep-config> <toolchain> <name> <version>", + + "\h|SYNOPSIS| + + \c{\b{brep-monitor --help}\n + \b{brep-monitor --version}\n + \b{brep-monitor} [<options>] <brep-config> <toolchain> [<toolchain>...]} + + \c{<toolchain> = <name>[\b{/}<version>]} + + \h|DESCRIPTION| + + \cb{brep-monitor} analyzes the \cb{brep} internal state and reports the + infrastructure issues printing their descriptions to \cb{stderr}. + + The specified \cb{brep} configuration file (<brep-config>) is used to + retrieve information required to access the databases and deduce the + expected behavior. Most of this information can be overridden via the + command line options. + + Currently, only delayed package builds for the specified toolchains are + reported. If toolchain version is omitted then all package builds with + this toolchain name are considered. + + \cb{brep-monitor} maintains its own state in the brep \cb{build} + database. In particular, it records timestamps of the reported package + build delays and optionally omits them from being reported again during + the timeout specified with the \cb{--report-timeout} option. + + By default, a brief report is printed. Use the \cb{--full-report} + option to obtain the full report (which may be large). + + Note that \cb{brep-monitor} expects the \cb{build} database schema to + have already been created using \l{brep-migrate(1)}." + } + + class monitor + { + "\h|OPTIONS|" + + std::size_t --build-timeout + { + "<seconds>", + "Time to wait (in seconds) before considering a package build as + delayed. If unspecified, the sum of \cb{brep}'s + \cb{build-normal-rebuild-timeout} and \cb{build-result-timeout} + configuration option values is used. Note also that an archived + package that is unbuilt is always considered delayed." + } + + std::size_t --report-timeout + { + "<seconds>", + "Time to wait (in seconds) before repeating a report of a package + build delay. By default there is no delay and all reports are + repeated." + } + + bool --full-report + { + "Print the list of delayed package builds rather than just their number + per build configuration." + } + + bool --clean + { + "Additionally clean the monitor state removing outdated information + related to non-existent packages, configurations, etc." + } + + // Note that the web service would normally logs in under a different + // user (and potentially switch the role afterwords) and so falling back + // to brep's user name and password wouldn't make much sense. + // + std::string --build-db-user|-u + { + "<user>", + "\cb{build} database user name. If unspecified, then operating system + (login) name is used." + } + + std::string --build-db-password + { + "<pass>", + "\cb{build} database password. If unspecified, then login without + password is expected to work." + } + + std::string --build-db-name|-n = "brep_package" + { + "<name>", + "\cb{build} database name. If unspecified, then \cb{brep}'s + \cb{build-db-name} configuration option value is used." + } + + std::string --build-db-host|-h + { + "<host>", + "\cb{build} database host name, address, or socket. If unspecified, + then \cb{brep}'s \cb{build-db-host} configuration option value is + used." + } + + std::uint16_t --build-db-port|-p + { + "<port>", + "\cb{build} database port number. If unspecified, then \cb{brep}'s + \cb{build-db-port} configuration option value is used." + } + + std::string --pager // String to allow empty value. + { + "<path>", + "The pager program to be used to show long text. Commonly used pager + programs are \cb{less} and \cb{more}. You can also specify additional + options that should be passed to the pager program with + \cb{--pager-option}. If an empty string is specified as the pager + program, then no pager will be used. If the pager program is not + explicitly specified, then \cb{brep-monitor} will try to use + \cb{less}. If it is not available, then no pager will be used." + } + + std::vector<std::string> --pager-option + { + "<opt>", + "Additional option to be passed to the pager program. See \cb{--pager} + for more information on the pager program. Repeat this option to + specify multiple pager options." + } + + bool --help {"Print usage information and exit."} + bool --version {"Print version and exit."} + }; + + "\h|EXIT STATUS| + + \dl| + + \li|\cb{0} + + Success.| + + \li|\cb{1} + + Fatal error.| + + \li|\cb{2} + + An instance of \cb{brep-monitor} or some other \cb{brep} utility is + already running. Try again.| + + \li|\cb{3} + + Recoverable database error. Try again.|| + " + } +} diff --git a/monitor/monitor.cxx b/monitor/monitor.cxx new file mode 100644 index 0000000..e04c5e1 --- /dev/null +++ b/monitor/monitor.cxx @@ -0,0 +1,766 @@ +// file : monitor/monitor.cxx -*- C++ -*- +// license : MIT; see accompanying LICENSE file + +#include <map> +#include <set> +#include <chrono> +#include <iostream> +#include <algorithm> // find_if() + +#include <odb/database.hxx> +#include <odb/transaction.hxx> +#include <odb/schema-catalog.hxx> + +#include <odb/pgsql/database.hxx> + +#include <libbutl/pager.mxx> +#include <libbutl/utility.mxx> // compare_c_string + +#include <libbbot/build-config.hxx> + +#include <libbrep/build.hxx> +#include <libbrep/common.hxx> +#include <libbrep/build-odb.hxx> +#include <libbrep/build-package.hxx> +#include <libbrep/build-package-odb.hxx> +#include <libbrep/database-lock.hxx> + +#include <mod/build-config.hxx> + +#include <monitor/module-options.hxx> +#include <monitor/monitor-options.hxx> + +using namespace std; +using namespace butl; +using namespace bbot; +using namespace odb::core; + +namespace brep +{ + // Operation failed, diagnostics has already been issued. + // + struct failed {}; + + static const char* help_info ( + " info: run 'brep-monitor --help' for more information"); + + static int + main (int argc, char* argv[]) + try + { + cli::argv_scanner scan (argc, argv); + options::monitor ops (scan); + + // Version. + // + if (ops.version ()) + { + cout << "brep-monitor " << BREP_VERSION_ID << endl + << "libbrep " << LIBBREP_VERSION_ID << endl + << "libbbot " << LIBBBOT_VERSION_ID << endl + << "libbpkg " << LIBBPKG_VERSION_ID << endl + << "libbutl " << LIBBUTL_VERSION_ID << endl + << "Copyright (c) " << BREP_COPYRIGHT << "." << endl + << "This is free software released under the MIT license." << endl; + + return 0; + } + + // Help. + // + if (ops.help ()) + { + pager p ("brep-monitor help", + false, + ops.pager_specified () ? &ops.pager () : nullptr, + &ops.pager_option ()); + + print_usage (p.stream ()); + + // If the pager failed, assume it has issued some diagnostics. + // + return p.wait () ? 0 : 1; + } + + // Parse the brep module configuration. + // + options::module mod_ops; + { + if (!scan.more ()) + { + cerr << "error: brep module configuration file is expected" << endl + << help_info << endl; + return 1; + } + + string f (scan.next ()); + + try + { + cli::argv_file_scanner scan (f, "" /* option */); + + // Parse the brep module options skipping those we don't recognize. + // + while (scan.more ()) + { + // Parse until an unknown option is encountered. + // + mod_ops.parse (scan, + cli::unknown_mode::stop, + cli::unknown_mode::stop); + + // Skip the unknown option, unless we are done. + // + if (scan.more ()) + { + // Skip the option name. + // + size_t l (scan.peek_line ()); + scan.skip (); + + // Skip the option value, if present. + // + // Note that here we rely on the configuration file having both + // the option name and its value on the same line. + // + if (scan.more () && scan.peek_line () == l) + scan.skip (); + } + } + } + catch (const cli::file_io_failure& e) + { + cerr << "error: unable to parse brep module configuration: " << e + << endl; + return 1; + } + catch (const cli::exception& e) + { + cerr << "error: unable to parse brep module configuration file '" << f + << "': " << e << endl; + return 1; + } + } + + if (!mod_ops.build_config_specified ()) + { + cerr << "warning: package building functionality is disabled" << endl; + return 0; + } + + // Parse the toolchains suppressing duplicates. + // + // Note that specifying a toolchain both with and without version doesn't + // make sense, so we fail if that's the case. + // + vector<pair<string, version>> toolchains; + + if (!scan.more ()) + { + cerr << "error: toolchain is expected" << endl << help_info << endl; + return 1; + } + + while (scan.more ()) + { + string s (scan.next ()); + + string tn; + version tv; + + try + { + size_t p (s.find ('/')); + + if (p == string::npos) + tn = move (s); + else + { + tn.assign (s, 0, p); + tv = version (string (s, p + 1)); + } + + bool dup (false); + for (const pair<string, version>& t: toolchains) + { + if (tn == t.first) + { + if (tv == t.second) + { + dup = true; + break; + } + + if (tv.empty () != t.second.empty ()) + { + cerr << "error: toolchain '" << tn << "' is specified both " + << "with and without version" << endl; + return 1; + } + } + } + + if (!dup) + toolchains.emplace_back (move (tn), move (tv)); + } + catch (const invalid_argument& e) + { + cerr << "error: invalid toolchain '" << s << "': " << e << endl; + return 1; + } + } + + // Parse buildtab. + // + build_configs configs; + + try + { + configs = parse_buildtab (mod_ops.build_config ()); + } + catch (const tab_parsing& e) + { + cerr << "error: unable to parse buildtab: " << e << endl; + return 1; + } + catch (const io_error& e) + { + cerr << "error: unable to read '" << mod_ops.build_config () << "': " + << e << endl; + return 1; + } + + // Create the database instance. + // + odb::pgsql::database db ( + ops.build_db_user (), + ops.build_db_password (), + (ops.build_db_name_specified () + ? ops.build_db_name () + : mod_ops.build_db_name ()), + (ops.build_db_host_specified () + ? ops.build_db_host () + : mod_ops.build_db_host ()), + (ops.build_db_port_specified () + ? ops.build_db_port () + : mod_ops.build_db_port ()), + "options='-c default_transaction_isolation=serializable'"); + + // Prevent several brep utility instances from updating the build database + // simultaneously. + // + database_lock l (db); + + // Check that the database schema matches the current one. + // + const string ds ("build"); + if (schema_catalog::current_version (db, ds) != db.schema_version (ds)) + { + cerr << "error: build database schema differs from the current one" + << endl + << " info: use brep-migrate to migrate the database" << endl; + return 1; + } + + // If requested, cleanup delays for package builds that are not expected + // anymore (build configuration is not present, etc). + // + if (ops.clean ()) + { + using config_map = map<const char*, + const build_config*, + compare_c_string>; + + config_map conf_map; + for (const build_config& c: configs) + conf_map[c.name.c_str ()] = &c; + + // Prepare the build delay prepared query. + // + // Query package build delays in chunks in order not to hold locks for + // too long. Sort the result by package version as a first priority to + // minimize number of queries to the package database. Note that we + // still need to sort by configuration and toolchain to make sure that + // build delays are sorted consistently across queries and we don't miss + // any of them. + // + using query = query<build_delay>; + using prep_query = prepared_query<build_delay>; + + // Specify the portion. + // + size_t offset (0); + + query q ("ORDER BY" + + query::id.package.tenant + "," + + query::id.package.name + + order_by_version (query::id.package.version, + false /* first */) + "," + + query::id.configuration + "," + + query::id.toolchain_name + + order_by_version (query::id.toolchain_version, + false /* first */) + + "OFFSET" + query::_ref (offset) + "LIMIT 100"); + + connection_ptr conn (db.connection ()); + + prep_query pq ( + conn->prepare_query<build_delay> ("build-delay-query", q)); + + // Cache the delayed build package object to reuse it in case the next + // delay refers to the same package (which is often the case due to the + // query result sorting criteria we use). + // + package_id pid; + shared_ptr<build_package> p; + + for (bool ne (true); ne; ) + { + transaction t (conn->begin ()); + + // Query delays. + // + auto delays (pq.execute ()); + + if ((ne = !delays.empty ())) + { + // Iterate over the build delays and cleanup the outdated ones. + // + for (const build_delay& d: delays) + { + config_map::const_iterator ci; + + bool cleanup ( + // Check that the toolchain is still used. + // + find_if (toolchains.begin (), toolchains.end (), + [&d] (const pair<string, version>& t) + { + return t.first == d.toolchain_name && + t.second == d.toolchain_version; + }) == toolchains.end () || + // + // Check that the build configuration is still present. + // + (ci = conf_map.find (d.configuration.c_str ())) == + conf_map.end ()); + + // Check that the package still present, is buildable and doesn't + // exclude the build configuration. + // + if (!cleanup) + { + if (d.id.package != pid) + { + pid = d.id.package; + p = db.find<build_package> (pid); + } + + cleanup = (p == nullptr || + !p->buildable || + exclude (p->builds, + p->constraints, + *ci->second, + configs.class_inheritance_map)); + } + + if (cleanup) + db.erase (d); + else + ++offset; + } + } + + t.commit (); + } + } + + // Collect and report delays as separate steps not to hold database locks + // while printing to stderr. Also we need to properly order delays for + // printing. + // + // Iterate through all possible package builds creating the list of delays + // with the following sort priority: + // + // 1: toolchain name + // 2: toolchain version (descending) + // 3: configuration name + // 4: tenant + // 5: package name + // 6: package version (descending) + // + // Such ordering will allow us to group build delays by toolchain and + // configuration while printing the report. + // + struct compare_delay + { + bool + operator() (const shared_ptr<const build_delay>& x, + const shared_ptr<const build_delay>& y) const + { + if (int r = x->toolchain_name.compare (y->toolchain_name)) + return r < 0; + + if (int r = x->toolchain_version.compare (y->toolchain_version)) + return r > 0; + + if (int r = x->configuration.compare (y->configuration)) + return r < 0; + + if (int r = x->tenant.compare (y->tenant)) + return r < 0; + + if (int r = x->package_name.compare (y->package_name)) + return r < 0; + + return x->package_version.compare (y->package_version) > 0; + } + }; + + set<shared_ptr<const build_delay>, compare_delay> delays; + { + connection_ptr conn (db.connection ()); + + // Prepare the buildable package prepared query. + // + // Query buildable packages in chunks in order not to hold locks for + // too long. + // + using pquery = query<buildable_package>; + using prep_pquery = prepared_query<buildable_package>; + + // Specify the portion. + // + size_t offset (0); + + pquery pq ("ORDER BY" + + pquery::build_package::id.tenant + "," + + pquery::build_package::id.name + + order_by_version (pquery::build_package::id.version, + false /* first */) + + "OFFSET" + pquery::_ref (offset) + "LIMIT 50"); + + prep_pquery ppq ( + conn->prepare_query<buildable_package> ("buildable-package-query", + pq)); + + // Prepare the package build prepared query. + // + // This query will only be used for toolchains that have no version + // specified on the command line to obtain the latest build across all + // toolchain versions. + // + using bquery = query<package_build>; + using prep_bquery = prepared_query<package_build>; + + build_id id; + const auto& bid (bquery::build::id); + + bquery bq ((equal<package_build> (bid.package, id.package) && + bid.configuration == bquery::_ref (id.configuration) && + bid.toolchain_name == bquery::_ref (id.toolchain_name)) + + "ORDER BY" + bquery::build::timestamp + "DESC" + "LIMIT 1"); + + prep_bquery pbq ( + conn->prepare_query<package_build> ("package-build-query", bq)); + + timestamp::duration build_timeout ( + ops.build_timeout_specified () + ? chrono::seconds (ops.build_timeout ()) + : chrono::seconds (mod_ops.build_normal_rebuild_timeout () + + mod_ops.build_result_timeout ())); + + timestamp now (system_clock::now ()); + + timestamp build_expiration (now - build_timeout); + + timestamp report_expiration ( + now - chrono::seconds (ops.report_timeout ())); + + for (bool ne (true); ne; ) + { + transaction t (conn->begin ()); + + // Query buildable packages (and cache the result). + // + auto bps (ppq.execute ()); + + if ((ne = !bps.empty ())) + { + offset += bps.size (); + + for (auto& bp: bps) + { + shared_ptr<build_package> p (db.load<build_package> (bp.id)); + + for (const build_config& c: configs) + { + if (exclude (p->builds, + p->constraints, + c, + configs.class_inheritance_map)) + continue; + + for (const pair<string, version>& t: toolchains) + { + id = build_id (p->id, c.name, t.first, t.second); + + // If the toolchain version is not specified then search for + // the latest build across all toolchain versions and search + // for a specific build otherwise. + // + shared_ptr<build> b; + + if (id.toolchain_version.empty ()) + { + auto pbs (pbq.execute ()); + + if (!pbs.empty ()) + b = move (pbs.begin ()->build); + } + else + b = db.find<build> (id); + + // Note that we consider a build as delayed if it is not + // completed in the expected timeframe. So even if the build + // task have been issued recently we may still consider the + // build as delayed. + // + timestamp bct (b != nullptr + ? b->completion_timestamp + : timestamp_nonexistent); + + // Create the delay object to record a timestamp when the + // package build could have potentially been started, unless + // it already exists. + // + shared_ptr<build_delay> d (db.find<build_delay> (id)); + + if (d == nullptr) + { + // If the archived package has no build nor build delay + // for this configuration, then we assume that the + // configuration was added after the package tenant has + // been archived and so the package could have never been + // built for this configuration. Thus, we don't consider + // this build as delayed and so skip it. + // + if (bp.archived && b == nullptr) + continue; + + // Use the build completion or build status change + // timestamp, whichever is earlier, as the build delay + // tracking starting point and fallback to the current time + // if there is no build yet. + // + timestamp pts ( + b == nullptr ? now : + bct != timestamp_nonexistent && bct < b->timestamp ? bct : + b->timestamp); + + d = make_shared<build_delay> (move (id.package.tenant), + move (id.package.name), + p->version, + move (id.configuration), + move (id.toolchain_name), + t.second, + pts); + db.persist (d); + } + + // Handle package builds differently based on their tenant's + // archive status. + // + // If the package is not archived then consider it as delayed + // if it is not (re-)built by the expiration time. Otherwise, + // consider it as delayed if it is unbuilt. + // + bool delayed; + + if (!bp.archived) + { + timestamp bts (bct != timestamp_nonexistent + ? bct + : d->package_timestamp); + + delayed = (bts <= build_expiration); + } + else + delayed = (bct == timestamp_nonexistent); + + // If the report timeout is not specified then report the + // delay unconditionally. Otherwise, report the active package + // build delay if the report timeout is expired and the + // archived package build delay if it was never reported. Note + // that fixing the building infrastructure won't help building + // an archived package, so reporting its build delays + // repeatedly is meaningless. + // + if (delayed && + (!ops.report_timeout_specified () || + (!bp.archived + ? d->report_timestamp <= report_expiration + : d->report_timestamp == timestamp_nonexistent))) + { + // Note that we update the delay objects persistent state + // later, after we successfully print the report. + // + d->report_timestamp = now; + delays.insert (move (d)); + } + } + } + } + } + + t.commit (); + } + } + + // Report package build delays, if any. + // + if (!delays.empty ()) + try + { + // Print the report. + // + cerr.exceptions (ostream::badbit | ostream::failbit); + + cerr << "Package build delays (" << delays.size () << "):" << endl; + + // Group the printed delays by toolchain and configuration. + // + const string* toolchain_name (nullptr); + const version* toolchain_version (nullptr); + const string* configuration (nullptr); + + // Print the delayed package build number per configuration rather than + // the packages themselves in the brief report mode. + // + size_t config_build_count (0); + + auto brief_config = [&configuration, &config_build_count] () + { + if (configuration != nullptr) + { + cerr << " " << *configuration << " (" << config_build_count + << ")" << endl; + + config_build_count = 0; + } + }; + + for (shared_ptr<const build_delay> d: delays) + { + // Print the toolchain, if changed. + // + if (toolchain_name == nullptr || + d->toolchain_name != *toolchain_name || + d->toolchain_version != *toolchain_version) + { + if (!ops.full_report ()) + brief_config (); + + if (toolchain_name != nullptr) + cerr << endl; + + cerr << " " << d->toolchain_name; + + if (!d->toolchain_version.empty ()) + cerr << "/" << d->toolchain_version; + + cerr << endl; + + toolchain_name = &d->toolchain_name; + toolchain_version = &d->toolchain_version; + configuration = nullptr; + } + + // Print the configuration, if changed. + // + if (configuration == nullptr || d->configuration != *configuration) + { + if (ops.full_report ()) + { + if (configuration != nullptr) + cerr << endl; + + cerr << " " << d->configuration << endl; + } + else + brief_config (); + + configuration = &d->configuration; + } + + // Print the delayed build package in the full report mode and count + // configuration builds otherwise. + // + if (ops.full_report ()) + { + // We can potentially extend this information with the archived flag + // or the delay duration. + // + cerr << " " << d->package_name << "/" << d->package_version; + + if (!d->tenant.empty ()) + cerr << " " << d->tenant; + + cerr << endl; + } + else + ++config_build_count; + } + + if (!ops.full_report ()) + brief_config (); + + // Persist the delay report timestamps. + // + transaction t (db.begin ()); + + for (shared_ptr<const build_delay> d: delays) + db.update (d); + + t.commit (); + } + catch (const io_error&) + { + return 1; // Not much we can do on stderr writing failure. + } + + return 0; + } + catch (const database_locked&) + { + cerr << "brep-monitor or some other brep utility is running" << endl; + return 2; + } + catch (const recoverable& e) + { + cerr << "recoverable database error: " << e << endl; + return 3; + } + catch (const cli::exception& e) + { + cerr << "error: " << e << endl << help_info << endl; + return 1; + } + catch (const failed&) + { + return 1; // Diagnostics has already been issued. + } + // Fully qualified to avoid ambiguity with odb exception. + // + catch (const std::exception& e) + { + cerr << "error: " << e << endl; + return 1; + } +} + +int +main (int argc, char* argv[]) +{ + return brep::main (argc, argv); +} diff --git a/tests/web/xhtml/buildfile b/tests/web/xhtml/buildfile index 6ddd5ae..ff683b9 100644 --- a/tests/web/xhtml/buildfile +++ b/tests/web/xhtml/buildfile @@ -1,7 +1,7 @@ # file : tests/web/xhtml/buildfile # license : MIT; see accompanying LICENSE file -include ../../../web/ +include ../../../web/xhtml/ -exe{driver}: {hxx cxx}{*} ../../../web/libus{web} +exe{driver}: {hxx cxx}{*} ../../../web/xhtml/libue{xhtml} exe{driver}: file{test.out}: test.stdout = true diff --git a/tests/web/xhtml/driver.cxx b/tests/web/xhtml/driver.cxx index ff554e4..a0135de 100644 --- a/tests/web/xhtml/driver.cxx +++ b/tests/web/xhtml/driver.cxx @@ -6,7 +6,7 @@ #include <libstudxml/serializer.hxx> -#include <web/xhtml.hxx> +#include <web/xhtml/serialization.hxx> using namespace std; using namespace xml; diff --git a/web/apache/log.hxx b/web/server/apache/log.hxx index 6609190..f7738ef 100644 --- a/web/apache/log.hxx +++ b/web/server/apache/log.hxx @@ -1,8 +1,8 @@ -// file : web/apache/log.hxx -*- C++ -*- +// file : web/server/apache/log.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_APACHE_LOG_HXX -#define WEB_APACHE_LOG_HXX +#ifndef WEB_SERVER_APACHE_LOG_HXX +#define WEB_SERVER_APACHE_LOG_HXX #include <httpd.h> // request_rec, server_rec #include <http_log.h> @@ -11,7 +11,7 @@ #include <cstdint> // uint64_t #include <algorithm> // min() -#include <web/module.hxx> +#include <web/server/module.hxx> namespace web { @@ -77,4 +77,4 @@ namespace web } } -#endif // WEB_APACHE_LOG_HXX +#endif // WEB_SERVER_APACHE_LOG_HXX diff --git a/web/apache/request.cxx b/web/server/apache/request.cxx index 4722b7f..a413081 100644 --- a/web/apache/request.cxx +++ b/web/server/apache/request.cxx @@ -1,7 +1,7 @@ -// file : web/apache/request.cxx -*- C++ -*- +// file : web/server/apache/request.cxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#include <web/apache/request.hxx> +#include <web/server/apache/request.hxx> #include <apr.h> // APR_SIZE_MAX #include <apr_errno.h> // apr_status_t, APR_SUCCESS, APR_E*, apr_strerror() @@ -38,7 +38,7 @@ #include <libbutl/optional.mxx> #include <libbutl/timestamp.mxx> -#include <web/mime-url-encoding.hxx> +#include <web/server/mime-url-encoding.hxx> using namespace std; using namespace butl; diff --git a/web/apache/request.hxx b/web/server/apache/request.hxx index 793a09d..bc105ec 100644 --- a/web/apache/request.hxx +++ b/web/server/apache/request.hxx @@ -1,8 +1,8 @@ -// file : web/apache/request.hxx -*- C++ -*- +// file : web/server/apache/request.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_APACHE_REQUEST_HXX -#define WEB_APACHE_REQUEST_HXX +#ifndef WEB_SERVER_APACHE_REQUEST_HXX +#define WEB_SERVER_APACHE_REQUEST_HXX #include <httpd.h> // request_rec, HTTP_*, OK, M_POST @@ -14,8 +14,8 @@ #include <ostream> #include <streambuf> -#include <web/module.hxx> -#include <web/apache/stream.hxx> +#include <web/server/module.hxx> +#include <web/server/apache/stream.hxx> namespace web { @@ -228,6 +228,6 @@ namespace web } } -#include <web/apache/request.ixx> +#include <web/server/apache/request.ixx> -#endif // WEB_APACHE_REQUEST_HXX +#endif // WEB_SERVER_APACHE_REQUEST_HXX diff --git a/web/apache/request.ixx b/web/server/apache/request.ixx index 3a1c01a..119fd2e 100644 --- a/web/apache/request.ixx +++ b/web/server/apache/request.ixx @@ -1,4 +1,4 @@ -// file : web/apache/request.ixx -*- C++ -*- +// file : web/server/apache/request.ixx -*- C++ -*- // license : MIT; see accompanying LICENSE file #include <http_protocol.h> // ap_*() diff --git a/web/apache/service.cxx b/web/server/apache/service.cxx index 1eeb65e..9fb23da 100644 --- a/web/apache/service.cxx +++ b/web/server/apache/service.cxx @@ -1,7 +1,7 @@ -// file : web/apache/service.cxx -*- C++ -*- +// file : web/server/apache/service.cxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#include <web/apache/service.hxx> +#include <web/server/apache/service.hxx> #include <apr_pools.h> // apr_palloc() @@ -18,8 +18,8 @@ #include <libbutl/utility.mxx> // function_cast() #include <libbutl/optional.mxx> -#include <web/module.hxx> -#include <web/apache/log.hxx> +#include <web/server/module.hxx> +#include <web/server/apache/log.hxx> using namespace std; using namespace butl; diff --git a/web/apache/service.hxx b/web/server/apache/service.hxx index aaf006e..ad54d2c 100644 --- a/web/apache/service.hxx +++ b/web/server/apache/service.hxx @@ -1,8 +1,8 @@ -// file : web/apache/service.hxx -*- C++ -*- +// file : web/server/apache/service.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_APACHE_SERVICE_HXX -#define WEB_APACHE_SERVICE_HXX +#ifndef WEB_SERVER_APACHE_SERVICE_HXX +#define WEB_SERVER_APACHE_SERVICE_HXX #include <apr_pools.h> // apr_pool_t #include <apr_hooks.h> // APR_HOOK_* @@ -15,9 +15,9 @@ #include <string> #include <cassert> -#include <web/module.hxx> -#include <web/apache/log.hxx> -#include <web/apache/request.hxx> +#include <web/server/module.hxx> +#include <web/server/apache/log.hxx> +#include <web/server/apache/request.hxx> namespace web { @@ -328,6 +328,6 @@ namespace web } } -#include <web/apache/service.txx> +#include <web/server/apache/service.txx> -#endif // WEB_APACHE_SERVICE_HXX +#endif // WEB_SERVER_APACHE_SERVICE_HXX diff --git a/web/apache/service.txx b/web/server/apache/service.txx index bda8e10..1b16d0b 100644 --- a/web/apache/service.txx +++ b/web/server/apache/service.txx @@ -1,4 +1,4 @@ -// file : web/apache/service.txx -*- C++ -*- +// file : web/server/apache/service.txx -*- C++ -*- // license : MIT; see accompanying LICENSE file #include <httpd.h> // APEXIT_CHILDSICK diff --git a/web/apache/stream.hxx b/web/server/apache/stream.hxx index ed0018e..77145af 100644 --- a/web/apache/stream.hxx +++ b/web/server/apache/stream.hxx @@ -1,8 +1,8 @@ -// file : web/apache/stream.hxx -*- C++ -*- +// file : web/server/apache/stream.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_APACHE_STREAM_HXX -#define WEB_APACHE_STREAM_HXX +#ifndef WEB_SERVER_APACHE_STREAM_HXX +#define WEB_SERVER_APACHE_STREAM_HXX #include <httpd.h> // request_rec, HTTP_* #include <http_protocol.h> // ap_*() @@ -13,7 +13,7 @@ #include <streambuf> #include <algorithm> // min(), max() -#include <web/module.hxx> // invalid_request +#include <web/server/module.hxx> // invalid_request namespace web { @@ -145,4 +145,4 @@ namespace web } } -#endif // WEB_APACHE_STREAM_HXX +#endif // WEB_SERVER_APACHE_STREAM_HXX diff --git a/web/buildfile b/web/server/buildfile index 0ca6cf6..26de70f 100644 --- a/web/buildfile +++ b/web/server/buildfile @@ -1,4 +1,4 @@ -# file : web/buildfile +# file : web/server/buildfile # license : MIT; see accompanying LICENSE file # This is currently part of the brep apache module but lives in a separate @@ -8,11 +8,8 @@ # import libs = libapr1%lib{apr-1} import libs += libapreq2%lib{apreq2} -import libs += libstudxml%lib{studxml} import libs += libbutl%lib{butl} -libus{web}: {hxx ixx txx cxx}{** -version} {hxx}{version} $libs - -hxx{version}: in{version} $src_root/manifest +libus{web-server}: {hxx ixx txx cxx}{**} $libs {hxx ixx txx}{*}: install = false diff --git a/web/mime-url-encoding.cxx b/web/server/mime-url-encoding.cxx index e202f08..fd1e4e8 100644 --- a/web/mime-url-encoding.cxx +++ b/web/server/mime-url-encoding.cxx @@ -1,7 +1,7 @@ -// file : web/mime-url-encoding.cxx -*- C++ -*- +// file : web/server/mime-url-encoding.cxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#include <web/mime-url-encoding.hxx> +#include <web/server/mime-url-encoding.hxx> #include <string> #include <iterator> // back_inserter diff --git a/web/mime-url-encoding.hxx b/web/server/mime-url-encoding.hxx index b9d2a76..34172a4 100644 --- a/web/mime-url-encoding.hxx +++ b/web/server/mime-url-encoding.hxx @@ -1,8 +1,8 @@ -// file : web/mime-url-encoding.hxx -*- C++ -*- +// file : web/server/mime-url-encoding.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_MIME_URL_ENCODING_HXX -#define WEB_MIME_URL_ENCODING_HXX +#ifndef WEB_SERVER_MIME_URL_ENCODING_HXX +#define WEB_SERVER_MIME_URL_ENCODING_HXX #include <string> @@ -29,4 +29,4 @@ namespace web bool query = true); } -#endif // WEB_MIME_URL_ENCODING_HXX +#endif // WEB_SERVER_MIME_URL_ENCODING_HXX diff --git a/web/module.hxx b/web/server/module.hxx index 5e9959e..beda73c 100644 --- a/web/module.hxx +++ b/web/server/module.hxx @@ -1,8 +1,8 @@ -// file : web/module.hxx -*- C++ -*- +// file : web/server/module.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_MODULE_HXX -#define WEB_MODULE_HXX +#ifndef WEB_SERVER_MODULE_HXX +#define WEB_SERVER_MODULE_HXX #include <map> #include <string> @@ -296,4 +296,4 @@ namespace web }; } -#endif // WEB_MODULE_HXX +#endif // WEB_SERVER_MODULE_HXX diff --git a/web/version.hxx.in b/web/version.hxx.in deleted file mode 100644 index ba51d44..0000000 --- a/web/version.hxx.in +++ /dev/null @@ -1,11 +0,0 @@ -// file : web/version.hxx.in -*- C++ -*- -// license : MIT; see accompanying LICENSE file - -#ifndef WEB_VERSION_HXX_IN -#define WEB_VERSION_HXX_IN - -#include <libstudxml/version.hxx> - -$libstudxml.check(LIBSTUDXML_VERSION, LIBSTUDXML_SNAPSHOT)$ - -#endif // WEB_VERSION_HXX_IN diff --git a/web/.gitignore b/web/xhtml/.gitignore index 426db9e..426db9e 100644 --- a/web/.gitignore +++ b/web/xhtml/.gitignore diff --git a/web/xhtml/buildfile b/web/xhtml/buildfile new file mode 100644 index 0000000..06dd34c --- /dev/null +++ b/web/xhtml/buildfile @@ -0,0 +1,10 @@ +# file : web/xhtml/buildfile +# license : MIT; see accompanying LICENSE file + +import libs = libstudxml%lib{studxml} + +./: {libue libus}{xhtml}: {hxx ixx txx cxx}{** -version} {hxx}{version} $libs + +hxx{version}: in{version} $src_root/manifest + +{hxx ixx txx}{*}: install = false diff --git a/web/xhtml-fragment.cxx b/web/xhtml/fragment.cxx index dbe0f0f..843db82 100644 --- a/web/xhtml-fragment.cxx +++ b/web/xhtml/fragment.cxx @@ -1,7 +1,7 @@ -// file : web/xhtml-fragment.cxx -*- C++ -*- +// file : web/xhtml/fragment.cxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#include <web/xhtml-fragment.hxx> +#include <web/xhtml/fragment.hxx> #include <string> #include <cassert> @@ -9,7 +9,7 @@ #include <libstudxml/parser.hxx> #include <libstudxml/serializer.hxx> -#include <web/xhtml.hxx> +#include <web/xhtml/serialization.hxx> using namespace std; using namespace xml; diff --git a/web/xhtml-fragment.hxx b/web/xhtml/fragment.hxx index 832d9eb..eab4335 100644 --- a/web/xhtml-fragment.hxx +++ b/web/xhtml/fragment.hxx @@ -1,4 +1,4 @@ -// file : web/xhtml-fragment.hxx -*- C++ -*- +// file : web/xhtml/fragment.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file #ifndef WEB_XHTML_FRAGMENT_HXX diff --git a/web/xhtml.hxx b/web/xhtml/serialization.hxx index 727ad5f..03e72ff 100644 --- a/web/xhtml.hxx +++ b/web/xhtml/serialization.hxx @@ -1,12 +1,12 @@ -// file : web/xhtml.hxx -*- C++ -*- +// file : web/xhtml/serialization.hxx -*- C++ -*- // license : MIT; see accompanying LICENSE file -#ifndef WEB_XHTML_HXX -#define WEB_XHTML_HXX +#ifndef WEB_XHTML_SERIALIZATION_HXX +#define WEB_XHTML_SERIALIZATION_HXX #include <libstudxml/serializer.hxx> -#include <web/version.hxx> +#include <web/xhtml/version.hxx> namespace web { @@ -355,4 +355,4 @@ namespace web } } -#endif // WEB_XHTML_HXX +#endif // WEB_XHTML_SERIALIZATION_HXX diff --git a/web/xhtml/version.hxx.in b/web/xhtml/version.hxx.in new file mode 100644 index 0000000..fe3e4e5 --- /dev/null +++ b/web/xhtml/version.hxx.in @@ -0,0 +1,11 @@ +// file : web/xhtml/version.hxx.in -*- C++ -*- +// license : MIT; see accompanying LICENSE file + +#ifndef WEB_XHTML_VERSION_HXX_IN +#define WEB_XHTML_VERSION_HXX_IN + +#include <libstudxml/version.hxx> + +$libstudxml.check(LIBSTUDXML_VERSION, LIBSTUDXML_SNAPSHOT)$ + +#endif // WEB_XHTML_VERSION_HXX_IN |