summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorBoris Kolpackov <boris@codesynthesis.com>2019-02-24 15:44:35 +0200
committerBoris Kolpackov <boris@codesynthesis.com>2019-02-24 15:44:35 +0200
commit757787725cffc24f6c4e353fefa63bf255007457 (patch)
treecce37b4ed8b88ac549d62aafb9fc5e04263bf514
parent135ce62b109cca705ca784942d5ef9dfc2f803dd (diff)
Updates for 0.9.0 releasev0.9.0
-rw-r--r--doc/.gitignore2
-rw-r--r--doc/bash-style.cli4
-rwxr-xr-xdoc/cli.sh5
-rw-r--r--doc/release.cli724
-rw-r--r--git/modules9
-rwxr-xr-xpublish90
-rw-r--r--release-bugfix-only.txt79
-rw-r--r--release.txt307
-rwxr-xr-xrep-test179
-rwxr-xr-xreview2
-rwxr-xr-xstage125
-rwxr-xr-xstage-pkg17
-rwxr-xr-xupgrade131
-rwxr-xr-xversion19
14 files changed, 981 insertions, 712 deletions
diff --git a/doc/.gitignore b/doc/.gitignore
index a5873dc..36afaee 100644
--- a/doc/.gitignore
+++ b/doc/.gitignore
@@ -1 +1,3 @@
build2-bash-style.xhtml
+build2-release.xhtml
+
diff --git a/doc/bash-style.cli b/doc/bash-style.cli
index 9e90e8a..d22cc23 100644
--- a/doc/bash-style.cli
+++ b/doc/bash-style.cli
@@ -9,7 +9,7 @@
// - Maximum <pre> line is 70 characters.
//
-"\h1|Contents|"
+"\h1|Table of Contents|"
"\$TOC$"
"
@@ -27,7 +27,7 @@ You just need to be clear on why you are doing it.
See also \l{https://google.github.io/styleguide/shell.xml Google's Bash Style
Guide} as well as \l{https://github.com/progrium/bashstyle Let's do Bash
right!}; we agree with quite a few (but not all) items in there. In particular,
-it provides a lot more rationale compared to this guide.
+the former provides a lot more rationale compared to this guide.
\h1#style|Style|
diff --git a/doc/cli.sh b/doc/cli.sh
index 44d311a..e737ee7 100755
--- a/doc/cli.sh
+++ b/doc/cli.sh
@@ -22,3 +22,8 @@ cli --generate-html --html-suffix .xhtml \
--html-prologue-file doc-prologue.xhtml \
--html-epilogue-file doc-epilogue.xhtml \
--output-prefix build2- bash-style.cli
+
+cli --generate-html --html-suffix .xhtml \
+--html-prologue-file doc-prologue.xhtml \
+--html-epilogue-file doc-epilogue.xhtml \
+--output-prefix build2- release.cli
diff --git a/doc/release.cli b/doc/release.cli
new file mode 100644
index 0000000..6639371
--- /dev/null
+++ b/doc/release.cli
@@ -0,0 +1,724 @@
+// file : doc/release.cli
+// copyright : Copyright (c) 2014-2018 Code Synthesis Ltd
+// license : MIT; see accompanying LICENSE file
+
+"\title=Release Process"
+
+// NOTES
+//
+// - Maximum <pre> line is 70 characters.
+//
+
+"\h1|Table of Contents|"
+"\$TOC$"
+
+"
+\h0#process|Process|
+
+Review the state and services list (currently on paper) for any new additions.
+Consider how/when they are updated/tested during the release process.
+
+
+\h1#stage|Stage|
+
+The staging repository is completely independent which means it must contain
+all the \c{build2} dependencies (plus a few extra packages for testing; see
+the \c{etc/stage} script for the complete list). This in turn means that if
+any of these dependencies are in the unreleased state, then they should go
+through the applicable steps in this section (e.g., updating of \c{NEWS}, etc)
+and then be queued and published (effectively released) as part of the
+\c{build2} release. Generally, however, we should strive to not unnecessarily
+bundle the release of dependencies with the release of \c{build2} to keep
+the process as streamlined as possible.
+
+\N|When unbundling the release of a dependency we need to remove its
+distribution from \c{etc/stage} and add the pre-distributed packages
+(for example, from \c{public}) to \c{staging/repository/1/}.|
+
+\b{Pre-Conditions:}
+
+\ul|
+
+\li|Current \c{stage} build is clean.||
+
+\h#copy|Update copyright if new year|
+
+ Use the \c{git/copyright} script. See the script header for instructions.
+
+\h#etc|Update \c{etc/git/modules}|
+
+ Review for any new modules. Remove \c{etc/} and \c{private/} from
+ \c{modules} to reduce noise during stat.
+
+\h#review|Review \c{@@}|
+
+ Review \c{@@} notes:
+
+ \
+ ./review.sh | less -R
+ \
+
+ At least look for \c{@@\ TMP}
+
+ \
+ ./review.sh | grep TMP
+ \
+
+\h#review-db|Review database schema changes|
+
+ Review database schema changelog differences in \c{bpkg}, \c{bdep}, and
+ \c{brep} compared to the previous release (tag) for any schema/data
+ migration that may be required (if something is missing, then it would also
+ have to be tested).
+
+\h#news|Update \c{NEWS} files|
+
+ Update in all projects (including dependencies) that will have a release
+ version.
+
+ See \c{etc/stage} for the complete list.
+
+
+\h#packaging|Release \c{packaging/} dependencies|
+
+ Release dependencies that have the snapshot version as described in
+ \c{private/build2-packaging.txt}.
+
+ \N|Maybe this should be done during queuing? Why do we release (but not
+ publish) these now and other dependencies later?|
+
+
+\h#dependencies|Finalize all other dependencies|
+
+ Make sure all other unreleased dependencies listed in \c{etc/stage} are
+ ready to be released. Effectively, the only remaining step should be to
+ change the version.
+
+ Do this in the dependency order and finish each off with:
+
+ \
+ git pull && bdep sync -fura && bdep test -ar
+ \
+
+
+\h#upgrade|Upgrade dependencies|
+
+ Upgrade all dependencies in the \c{build2} toolchain build:
+
+ \
+ etc/upgrade
+ \
+
+
+\h#hello|Update \c{hello/} projects|
+
+ These projects should closely track the output of \c{bdep-new(1)}. The
+ recommended procedure is to generate a new project with the same name and
+ then review/resolve the differences with \c{diff\ -ru}.
+
+ Every change in these projects must be accompanied by a revision increment
+ and a re-tag. This should be managed with \c{bdep-release(1)}:
+
+ \
+ # make changes
+ git add .
+ bdep release --revision --show-push
+ # review commit
+ git push ...
+ \
+
+ Once done, run the \c{intro} scripts and review any changes in the output
+ (this information will be helpful on the next step):
+
+ \
+ cd etc
+
+ ./intro2-tldr 2>&1 | tee intro2-tldr.out
+ diff -u intro2-tldr.orig intro2-tldr.out # Or use gitk.
+ mv intro2-tldr.out intro2-tldr.orig
+
+ ./intro2-tour 2>&1 | tee intro2-tour.out
+ diff -u intro2-tour.orig intro2-tour.out # Or use gitk.
+ mv intro2-tour.out intro2-tour.orig
+ \
+
+\h#doc|Review documentation|
+
+ Review the following documentation for (1) sample output changes and (2)
+ still being relevant/making sense.
+
+ \N|Ideally this should be done during development but it's easy to forget.
+ Also, check if there is any new documentation that has been added that is
+ not on the below list.|
+
+ \ul|
+
+ \li|Install guide: 1 & 2.|
+
+ \li|Toolchain introduction: 1 & 2 (use \c{intro} script output).|
+
+ \li|Introduction in the build system manual: 1 (uses \c{bdep-new(1)}
+ output).|
+
+ \li|Testscript manual: 1.||
+
+
+\h#submod|Update all submodules|
+
+ Make sure all working trees are clean. Review \c{sub_modules} in
+ \c{etc/git/modules} for any missing new modules, then:
+
+ \N|\c{etc/} and \c{private/} need to be committed manually.|
+
+ \
+ ./modup.sh
+ ./commit.sh # If changes.
+ ./push.sh
+
+ cd build2-toolchain
+
+ git submodule update --remote --checkout
+ git submodule foreach git submodule update --init --recursive
+
+ git status # There should only be 'new commits'; see README-GIT
+
+ git commit -a -m \"Update submodules\"
+ git push
+ \
+
+\h#stage-machines|Update \c{stage} \c{buildtab}s and build machines|
+
+ Review \c{stage} \c{buildtab} for any configurations to drop (for example,
+ an intermediate version of a compiler).
+
+ Based on these changes update \c{stage} CI \c{buildtab}, which is a subset
+ of the \c{stage} configurations (and is a base for the \c{queue}/\c{public}
+ configurations).
+
+ Review deployed machines against the updated \c{stage} \c{buildtab} and
+ remove those that are no longer used:
+
+ \
+ cd private/buildos/
+
+ ./ls-machines -c stage -c devel
+
+ ~/work/buildos/remove-machine <host> <machine>
+ \
+
+ Also review deployed machines against the latest available versions and
+ upgrade those that are not the latest:
+
+ \
+ cd private/buildos/
+
+ ./ls-machines -l \"/btrfs/$(whoami)/machines/default/\"
+ ./ls-machines -c stage -c devel
+
+ ~/work/build2/buildos/upload-machine <host> .../new-ver .../old-ver
+ \
+
+
+\h#restage|Restage|
+
+ Confirm \c{stage} and \c{devel} \c{buildos} images, hardware-specific
+ configurations are the same.
+
+ Review \c{staging/0/} and \c{staging/repository/1/} for anything stray.
+
+ Update all submodules in \c{build2-toolchain}:
+
+ \
+ git submodule update --remote --checkout
+ \
+
+ Restage with \c{baseutils}/\c{mingw} regeneration:
+
+ \
+ etc/stage -b
+ \
+
+ Upgrade \c{brep} on \c{stage} and sync latest \c{buildtab}s.
+
+ Verify \c{stage} build is clean, nothing is unbuilt.
+
+
+\h#install-stage|Test install scripts|
+
+ Test \l{https://stage.build2.org/0/ \c{stage} install scripts}, including
+ upgrading, as described in \c{private/install/testing.txt}.
+
+
+\h1#queue|Queue|
+
+\b{Pre-Conditions:}
+
+\ul|
+
+\li|Final \c{stage} build is clean.|
+
+\li|Build with the \c{queue} toolchain is inactive.||
+
+
+\h#queue-repo|Review \c{queue} repository|
+
+ Pull, review, and clean the \c{queue} \c{git} repository for any stale
+ packages or anything (ownership, etc) that may conflict with packages being
+ released.
+
+ Also clean toolchain distribution:
+
+ \
+ rm -rf cppget.org/queue/0/*
+ \
+
+\h#version-release|Change to release version|
+
+ Change to the release version in all the packages being released and tag
+ (see \c{etc/stage} for the list). Use \c{bdep-release(1)} unless a custom
+ versioning/tagging scripts are used:
+
+ \
+ bdep release --no-open --show-push [--alpha|--beta]
+ # review commit
+ git push ...
+ \
+
+ Do this in the dependency order and finish each off with:
+
+ \
+ bdep sync -fura && bdep update
+ \
+
+ For the \c{build2} packages (commit but don't push after each step if
+ required):
+
+ \ul|
+
+ \li|Close schema versions in \c{bpkg}, \c{bdep}, and \c{brep}.|
+
+ \li|Change \c{BUILD2_STAGE} in \c{build2/build2/config.hxx.in} to \c{false}.|
+
+ \li|Change version by updating (including with new modules) and then
+ executing:
+
+ \
+ etc/version
+ ./commit.sh
+ git -C build2-toolchain commit --amend # \"Change version to X.Y.Z\"
+ \
+
+ |
+
+ \li|Tag by executing \c{tag.sh\ <version>}.|
+
+ \li|Regenerate documentation in each package.|
+
+ \li|Upgrade all dependencies in configure-only mode by executing
+ \c{etc/upgrade\ -c}.|
+
+ \li|Trigger regeneration of version files (might require several runs
+ to \"close off\"):
+
+ \
+ BDEP_SYNC=0 b --match-only ~/work/build2/builds/gcc7-asan/
+ \
+
+ If using GCC prior to 8 then might also need explicit version target
+ upgrades:
+
+ \
+ BDEP_SYNC=0 b ~/work/build2/builds/gcc7-asan/.../hxx{version}
+ \
+
+ |
+
+ \li|Regenerate ODB in relevant packages passing upgraded configuration
+ path explicitly (\c{bdep} is not runnable):
+
+ \
+ ./odb.sh ~/work/build2/builds/gcc7-asan/
+ \
+
+ |
+
+ \li|Finish upgrading all dependencies by executing in the upgraded
+ configuration:
+
+ \
+ BDEP_SYNC=0 b ~/work/build2/builds/gcc7-asan/
+ \
+
+ ||
+
+ Verify key tests pass (in particular, the \c{bdep} tests will now be running
+ against \c{public} services):
+
+ \
+ b test: build2/ bpkg/ bdep/
+ \
+
+ \N|We could have queued after this step before preparing
+ \c{build2-toolchain}. However, splitting this seems to only increase
+ complexity without any major benefits (it's not hard to update submodules
+ and regenerated \c{build2-toolchain} if there are any issues with
+ packages). Plus we can start testing install scripts, etc.|
+
+ Next push the above changes and update all submodules in
+ \c{build2-toolchain}:
+
+ \
+ ./push.sh
+
+ cd build2-toolchain
+
+ git submodule update --remote --checkout
+
+ git status # There should only be 'new commits'; see README-GIT
+
+ git commit -a -m \"Update submodules\"
+ \
+
+ As well as (commit after each step if required):
+
+ \ul|
+
+ \li|Regenerate documentation in each package inside as well as in
+ \c{build2-toolchain} itself.|
+
+ \li|Update ODB by copying relevant files from the previous step (trust
+ me, this is the easy way).|
+
+ \li|Change \c{BUILD2_REPO} in \c{build2-toolchain} build scripts to
+ \c{queue}.||
+
+ Finally, push the changes:
+
+ \
+ git push
+ \
+
+\h#queuing|Queue|
+
+ Prepare packages and the toolchain distribution:
+
+ \
+ etc/stage -q -b
+ \
+
+ Sort non-alpha packages from \c{cppget.org/queue/1/alpha/} into appropriate
+ sections (we could probably automate this similar to \c{bdep-release(1)}).
+
+ Note also that we assume all the packages already have the corresponding
+ ownership information either in \c{queue} or \c{public}. However, if any
+ new packages were added, then that will have to be added as well.
+
+ Commit and push queue repository:
+
+ \
+ cd cppget.org/queue/
+ git add .
+ git ci -m \"Queue build2 toolchain X.Y.Z\"
+ git push
+ \
+
+\h#build-public|Verify queued packages build with \c{public}|
+
+ This makes sure that the new version can be built with the old toolchain.
+
+ While all the packages should build (except perhaps \c{libhello} which,
+ being based on the latest \c{bdep-new(1)} output, may use new features),
+ some tests may fail. In particular, we will be running \c{bpkg} tests using
+ a previous version of the build system and \c{bdep} tests also using a
+ previous version of \c{bpkg}. None of that is or will be supported. So
+ for now we manually review all the failed builds for anything suspicious
+ and in the future we may skip tests if the tool versions mismatch.
+
+
+\h#install-queue|Test install scripts|
+
+ Test \l{https://download.build2.org/queue/ \c{queue} install scripts}. Here
+ we just smoke-test each script on its \"primary\" platform and make sure
+ \c{queue} URLs/repositories are used.
+
+
+\h#upgrade-brep-bpub|Upgrade \c{brep} and \c{bpub} on \c{cppget.org}|
+
+ Upgrade \c{brep} and \c{bpub} using the queued toolchain according
+ to \c{private/brep-bpub-upgrade.txt}.
+
+ After upgrade, drop all package builds for \c{public} and \c{queue} and
+ verify there are no regressions. This makes sure that the \c{brep} services
+ in the new version are usable with the old toolchain. If there are issues,
+ they have to be fixed by publishing/queuing revisions.
+
+ \N|The \c{bdep} tests that exercise CI and publishing services do it in the
+ simulation mode. To properly test that these services are compatible with
+ the old version of the toolchain we would need to CI/publish a real test
+ package manually.|
+
+
+\h#start-queue|Start \c{queue} builds|
+
+ Update \c{queue} \c{buildtab} based on the \c{stage} CI \c{buildtab}
+ (normally just a copy sans the sanitized toolchain configurations).
+
+ Adjust \c{stage} and \c{devel} build host configurations to enable the
+ \c{queue} toolchain. Shift most instances from \c{stage} to \c{queue}
+ in the hardware class-specific configurations. Regenerate affected
+ configurations and reboot build hosts:
+
+ \
+ cd private/buildos/
+
+ ./gen-config stage <class>
+ ./gen-config devel <class>
+
+ ./po-hosts -r -c stage -c devel
+ \
+
+ Verify both \c{queue} and \c{public} builds with the queued toolchain,
+ fixing any breakages with revisions, moving to legacy, and/or queuing
+ replacements. We can only proceed further once we have a \"resolution\"
+ for every (newly) broken package.
+
+
+\h#stop-queue|Stop \c{queue} builds|
+
+ Adjust \c{stage} and \c{devel} build host configurations to disable the
+ \c{queue} toolchain (comment out). Regenerate affected configurations and
+ reboot build hosts as on the previous step.
+
+\h1#public|Public|
+
+\b{Pre-Conditions:}
+
+\ul|
+
+\li|A \"resolution\" is queued or published for every (newly) broken package.||
+
+\h#public-machines|Update \c{public} \c{buildtab}s and build machines|
+
+ Poweroff the old set of \c{public} build hosts:
+
+ \
+ ./po-hosts -c public
+ \
+
+ Update \c{public} \c{buildtab}s based on the \c{queue} \c{buildtab}
+ (normally just a copy).
+
+ Adjust build host configurations and add/remove new/old machines.
+
+ Replace the \c{public} \c{buildos} image on \c{build-cache} with the
+ one for \c{stage}.
+
+ Comment out the \c{public} toolchain in the build host configuration
+ (effectively making it a no-toolchain configuration) and power on the new
+ set of \c{public} build hosts.
+
+ Review deployed machines against the updated \c{public} \c{buildtab} and
+ remove those that are no longer used:
+
+ \
+ cd private/buildos/
+
+ ./ls-machines -c public
+
+ ~/work/buildos/remove-machine <host> <machine>
+ \
+
+ Also review deployed machines against the latest available versions and
+ upgrade those that are not the latest:
+
+ \
+ cd private/buildos/
+
+ ./ls-machines -l \"/btrfs/$(whoami)/machines/default/\"
+ ./ls-machines -c public
+
+ ~/work/build2/buildos/upload-machine <host> .../new-ver .../old-ver
+ \
+
+ Uncomment the \c{public} toolchain in the build host configuration. The
+ only remaining step is to reboot (not yet):
+
+ \
+ ./po-hosts -r -c public
+ \
+
+\h#pub-dist|Publish distribution|
+
+ Change \c{BUILD2_REPO} in \c{build2-toolchain} build scripts to \c{public}
+ and publish the distribution (this also cleans/disables the \c{queue}
+ toolchain):
+
+ \
+ etc/stage -p
+ \
+
+\h#pub-pkg|Publish packages|
+
+ Move packages (manually for now) from the \c{queue} to \c{public} \c{git}
+ repository, including ownership information. Move old/replaced/FTB
+ packages either to legacy or delete.
+
+ Commit everything (see the commit log for procedure @@ this needs
+ automation) and push (which should trigger auto-publish). Note: commit and
+ push both \c{queue} and \c{public} \c{git} repositories.
+
+ Note that once published, the existing install instructions/download
+ links are no longer usable, so do not linger.
+
+\h#start-public|Start \c{public} builds|
+
+ Reboot \c{public} build hosts. They should next bootstrap and proceed
+ with building all packages.
+
+\h#install-public|Test install scripts|
+
+ Test \l{https://download.build2.org/ \c{public} install scripts}. Here
+ we just smoke-test each script on its \"primary\" platform and make sure
+ \c{public} URLs/repositories are used.
+
+\h#web|Update web|
+
+ \ul|
+
+ \li|Write release notes, use placeholder for announcement URL (or guess).
+ Add link from the \c{doc.cli}. Add blog entry.|
+
+ \li|Add any new FAQ entries or any other updates (main page, etc). Any new
+ documentation to link from the Doc page?|
+
+ \li|Update the Download page.
+
+ \
+ cat `ls -1 cppget.org/public/0/X.Y.Z/*.sha256`
+ \
+
+ |
+
+ \li|Update the Install page.
+
+ \
+ cat cppget.org/public/0/toolchain.sha256
+ \
+
+ |
+
+ \li|Regenerate documentation in \c{private/} for all hosts:
+
+ \
+ cd private/
+
+ cd <host>/www/
+ ./cli.sh
+ \
+
+ |
+
+ \li|Test locally, publish, and test remote:
+
+ \
+ cd private/
+ ./publish --dry-run
+ ./publish
+ \
+
+ ||
+
+
+\h#ann|Announce|
+
+ \ul|
+
+ \li|Write and send the announcement to the mailing lists.
+
+ \
+ cat `ls -1 cppget.org/public/0/X.Y.Z/*.sha256`
+ \
+
+ Add \c{reply-to:} header for \c{users@} announcement.|
+
+ \li|Patch (or verify) the announcement URL in release notes, re-publish.|
+
+ \li|Announce on reddit, slack, and other relevant places.||
+
+\h#tag|Commit, tag, and push the rest.|
+
+ Tag \c{build2-toolchain}:
+
+ \
+ cd build2-toolchain
+ git tag -a vX.Y.Z -m \"Tag version X.Y.Z\"
+ git push --follow-tags
+ \
+
+ Add \c{etc/} and \c{private/} back to \c{modules} in \c{etc/git/modules}
+ (essentially reverse \l{#etc Update \c{etc/git/modules}}).
+
+ Finalize changes, commit, tag, and push \c{style/}, \c{etc/}, and
+ \c{private/}.
+
+ Snapshot \c{buildos} subvolumes:
+
+ \
+ btrfs subvolume snapshot buildos-3 buildos-3-X.Y.Z
+ btrfs subvolume snapshot buildos-6 buildos-6-X.Y.Z
+ \
+
+\h1#reopen|Reopen|
+
+\h#version-snapshot|Change to snapshot version|
+
+ Change to the snapshot version in all the released packages. Use
+ \c{bdep-release(1)} unless a custom versioning script is used:
+
+ \
+ bdep release --open --show-push [--open-*]
+ # review commit
+ git push ...
+ \
+
+ Essentially, the same steps as in \l{#version-release Change to
+ release version} (but no tagging).
+
+\h#stage-machines-reopen|Update \c{stage} \c{buildtab}s and build machines|
+
+ Essentially, the same steps as in \l{#public-machines Update \c{public}
+ \c{buildtab}s and build machines} but for stage. Some differences:
+
+ Clean \c{buildtab}s (both \c{stage} and CI) by removing no longer relevant
+ configurations, moving some to \c{legacy}, etc.
+
+ More generally, this is the time to do sweeping changes such as renaming
+ machines/configurations, adjusting classes, etc. This is also the time to
+ rebalance machines across available hosts.
+
+\h#restage-reopen|Restage|
+
+ Make symlinks for the new version in \c{private/baseutils} (for both
+ \c{baseutils} and \c{mingw}; the idea is that we will start with those and
+ maybe upgrade later).
+
+ Then cleanup and restage:
+
+ \
+ rm -r staging/0/*
+ rm -r staging/repository/1/*/
+
+ etc/stage -b
+ \
+
+\h#start-stage|Start \c{stage} builds|
+
+ Reboot \c{stage} build hosts. They should next bootstrap and proceed with
+ building all the staged packages. Make sure all the builds of the new
+ development snapshot are successful and there is nothing unbuilt.
+
+\h#commit-reopen|Commit and push \c{etc/} and \c{private/}.|
+
+ Commit and push changes to \c{etc/} and \c{private/}.
+
+"
diff --git a/git/modules b/git/modules
index a793829..d138b98 100644
--- a/git/modules
+++ b/git/modules
@@ -23,10 +23,10 @@ modules="$modules buildos"
modules="$modules msvc-linux"
modules="$modules openssl-agent"
-#modules="$modules etc"
-#modules="$modules private"
+modules="$modules etc"
+modules="$modules private"
-# We don't tag git/, etc/, and private/.
+# We don't tag git/, and etc/, private/, build2-toolchain/ are tagged manually.
#
tag_modules=" \
libbutl \
@@ -42,13 +42,12 @@ bbot \
libstd-modules \
msvc-linux \
openssl-agent \
-build2-toolchain \
buildos"
# Submodule update in build2-toolchain has to be done manually.
-# As well as in private.
#
sub_modules=" \
+etc \
private \
libbutl \
libbutl.bash \
diff --git a/publish b/publish
deleted file mode 100755
index e9f658f..0000000
--- a/publish
+++ /dev/null
@@ -1,90 +0,0 @@
-#! /usr/bin/env bash
-
-# Publish build2 to build2.org/cppget.org (and brep.cppget.org).
-#
-# The distribution is taken from cppget.org/0/ and packages from
-# cppget.org/repository/.
-#
-# -d
-# Only publish the distribution.
-#
-# -p
-# Only publish the packages.
-#
-usage="$0 [options] -- [<rsync-options>]"
-
-owd=`pwd`
-trap "{ cd $owd; exit 1; }" ERR
-set -o errtrace # Trap in functions.
-
-function info () { echo "$*" 1>&2; }
-function error () { info "$*"; exit 1; }
-
-pkg_only=
-dist_only=
-
-while [ $# -gt 0 ]; do
- case $1 in
- -d)
- dist_only=true
- shift
- ;;
- -p)
- pkg_only=true
- shift
- ;;
- --)
- shift
- break
- ;;
- *)
- error "unexpected $1"
- ;;
- esac
-done
-
-if [ -z "$pkg_only" ]; then
- v="$(sed -n -re 's/^version: ([^.]+\.[^.]+\.[^-]+(-[ab]\.[^.+]+)?).*$/\1/p' build2-toolchain/manifest)"
-
- if [ -z "$v" ]; then
- error "unable to extract version build2-toolchain/manifest"
- fi
-fi
-
-
-function sync ()
-{
- if [ -z "$pkg_only" ]; then
- info "build2.org:"
-
- rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \
-"${@}" "cppget.org/0/$v/" "build2.org:/var/www/download.build2.org/public/$v/"
-
- rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \
-"${@}" "cppget.org/0/toolchain.sha256" "build2.org:/var/www/download.build2.org/public/"
- fi
-
- if [ -z "$dist_only" ]; then
- info "cppget.org:"
- etc/rep-publish cppget.org/repository/1/ cppget.org:/var/bpkg/1/ "${@}"
-
- info "brep.cppget.org:"
- etc/rep-publish cppget.org/repository/1/ lists.cppget.org:/var/bpkg/1/ \
-"${@}"
- fi
-}
-
-sync --dry-run "${@}"
-
-r=
-while [ -z "$r" ]; do
- read -r -p "Continue? [yes/no]: " r
-
- case "$r" in
- yes) ;;
- no) exit 0 ;;
- *) r= ;;
- esac
-done
-
-sync --progress "${@}"
diff --git a/release-bugfix-only.txt b/release-bugfix-only.txt
deleted file mode 100644
index f8bb098..0000000
--- a/release-bugfix-only.txt
+++ /dev/null
@@ -1,79 +0,0 @@
-- Create X.Y branch if first bugfix release
-
- git branch X.Y X.Y.0
- git checkout X.Y
-
-- See if makes sense to cherry-pick other fixes/changes
-
-- Update NEWS if anything major, commit
-
-- Update version in manifest, cli.sh, commit and push:
-
- git ci -a -m "Bump version to X.Y.N"
- git push
-
-- Regenerate odb.sh, cli.sh
-
-- If need to rebuild build2-toolchain:
-
- - Create branch as above, then:
-
- git submodule update --checkout
-
- - Add the branch value for each entry in .gitmodules:
-
- branch = X.Y
-
- - Then checkout and commit updated submodules:
-
- git submodule update --remote <submodule>
-
- - Update version in manifest as above
-
- - Regenerate odb.sh (may need to checkout old version or copy old),
- cli.sh as above.
-
- - Stage and diff with previous version for sanity check.
-
-- Dist to queue:
-
- etc/stage-pkg -q -d -c <group> <proj>
-
- If need to dist build2-toolchain, then rename X.Y.N-1 to X.Y.N in
- private/baseutils, then:
-
- etc/stage -b -p
-
- Verify build2-toolchain works by building locally.
-
-- @@ TODO: Test queue.
-
-- Move package from queue to the appropriate repository, normally
- replacing the old package.
-
-- Regenerate the repository and publish (remove -p if also publishing
- build2-toolchain):
-
- cd cppget.org
- git -C repository add .
- git -C repository status
- ./update
-
- cd ..
- etc/publish -p
-
-- Update download links/checksums if changed build2-toolchain.
-
-- Tag the bugfix release:
-
- git tag -a X.Y.Z -m "Tag version X.Y.Z"
-
- Also in build2-toolchain if applicable.
-
-- Commit cppget.org/repository/ (see history for procedure)
-
-- Commit private if changed build2-toolchain.
-
-- Write and send announcements, remember to include checksum.
-
-- Switch back to master and regenerate cli.sh/odb.sh
diff --git a/release.txt b/release.txt
deleted file mode 100644
index 350bbaf..0000000
--- a/release.txt
+++ /dev/null
@@ -1,307 +0,0 @@
-@@ Need to sync hello/* projects with latest bdep-new output (and re-tag)
-
-@@ No upgrade testing
-
-- Add new bot machines/configurations/options from stage to queue to public
-
- @@ See paper in Build OS filder.
-
- @@ Might have to be done at release if things are incompatible. Maybe
- better/cleaner to always do this at release.
-
-- Replace 0.7.0 and 0.7 in this document with the new version.
-
-- Remove etc/ and private/ from modules in etc/git/modules to reduce
- noise during stat. Also, review for any new modules.
-
-- Review '@@' items [private/ excluded, update with new modules, at least
- look for @@ TMP; also check the review script for any new modules]:
-
- etc/review | less -R
-
-- Identify packages that will be released (see etc/stage). They should all be
- staged and built successfully, including submodule-updated build2-toolchain
- and baseutils/mingw. Review stageing/0/ and staging/1/ for anything stray.
-
-- Update and test local builds:
-
- b test: build2/ bpkg/ bdep/ brep/ bbot/ libbutl/ libbpkg/ libbbot/
- b msvc-linux/
-
-- Stage hello projects and test:
-
- cd hello
- ./foreach-git pull
- ./foreach-git stat
- ./stage
-
- cd ..
- etc/intro -s `pwd`/hello/repository/1/
- etc/intro -s https://hello.stage.build2.org/1
-
- - Make sure all the bot builds are successful (drop build db).
-
-- Test bootstrap build2-toolchain using scripts/batch files (bot builds
- are using makefile). Need to test build.sh, build-*.bat. Now test via
- automated install files: see private/install/testing.txt for details.
-
-- Update NEWS files in all project that will have non-pre-release version.
-
-- Review documentation for (1) sample output changes and (2) still being
- relevant/making sense:
-
- - Testscript manual (for 1, see hello/hello-testscript).
-
- - Install guide for (for 1 & 2).
-
- - Introduction for (for 1 & 2). Use intro script to get output:
-
- ./intro2-tldr 2>&1 | tee intro2-tldr.out
- ./intro2-tour 2>&1 | tee intro2-tour.out
-
-- Change to final versions for all packages being released, some by hand,
- some with version scripts (see etc/stage for list + msvc-linux).
-
- - Close schema versions. Review schema changlog difference from previous
- release (tag) for any data migration that may be required (would also
- need to test this).
-
- - Push and update all submodules in buil2-toolchain
-
- - Regenerate all odb & docs (in both packages and inside build2-toolchain)
-
- - Bootstrap then update and test local
-
- cd build2
- b clean
- git cout build2/version.hxx build2/b-options.?xx
- ./bootstrap.sh g++-6
-
- cd ..
- b-boot '{clean update}(libbutl/ build2/)'
-
- b clean: bpkg/ bdep/ brep/ bbot/ libbpkg/ libbbot/ msvc-linux/
-
- Then the local test step above.
-
- - Cleanup staging repo:
-
- rm -r staging/0/*
- rm -r staging/repository/1/*/
-
- - Restage with -b:
-
- etc/stage -b
-
- - Check /0/ and /1/ for anything stray
-
- - restage hello (libstd-modules)
-
-- Upgrade brep on cppget.org (queue and main repo) using stage
-
-- Queue packages:
-
- - Change BUILD2_REPO in build2-toolchain build scripts to queue
-
- - Change build.version.stage in build2/build2/context.cxx to false.
-
- - Cleanup queue repo:
-
- rm -rf cppget.org/0/*
- rm -rf cppget.org/repository/1/queue/*/
-
- - Queue
-
- etc/stage -q -b
-
- - check /0/ and /1/ for anything stray
-
- @@ Add extra packages (cli, libcutl, mysql/mariadb).
-
- - Make sure bot builds are successful
-
-- Upgrade brep on build2.org (hello) using queue
-
-- Publish and test hello
-
- cd hello
- ./foreach-git stat
- ./stage -p
-
- cd ..
- etc/intro -s https://build2.org/pkg/1/hello
-
- - Drop build database (package versions do not change)
-
- - Make sure bot builds are successful
-
-- Publish and test toolchain
-
- - Change BUILD2_REPO in build2-toolchain build script to public
-
- - Regenerate build2-toolchain package in cppget.org/0/
-
- etc/stage -p
-
- @@ Remove *-queue install scripts.
-
- - Move packages (manually for now):
-
- - old/replaced/FTB packages either to legacy or delete
-
- - from queue to appropriate repositories (don't override existing)
-
- - Regenerate the repository
-
- cd cppget.org
- git -C repository add .
- git -C repository status # Review changes.
- ./update
-
- - Disable the queue toolchain on bbots (moved packages from queue/),
- delete distribution
-
- echo disabled | ssh build2.org tee >/dev/null \
- /var/www/download.build2.org/public/queue/toolchain.sha256
-
- ssh build2.org rm -r /var/www/download.build2.org/public/queue/*/
-
- - Publish everything to build2.org/cppget.org
-
- etc/publish
-
- - Make sure bot builds are successful, drop hello build database again.
-
- @@ First test automated install scripts.
-
-- Release
-
- - Update submodules (style/) in private/.
-
- - Write release notes, use placeholder for announcement URL (or guess).
- Add link from the doc page.
-
- - Any new FAQ entries or any other updates (main page, etc)? Any new
- documentation to link from the doc page?
-
- - Update download page.
-
- cat `ls -1 *.sha256`
-
- - Update install page.
-
- cat toolchain.sha256
-
- - Regenerate documentation (./cli.sh) for all hosts.
-
- - Test locally, publish, test production
-
- cd private/
- ./publish --dry-run
- ./publish
-
- - Write and send the announcement to mailing lists.
-
- cat `ls -1 *.sha256`
-
- - Remember upgrade instructions.
-
- - Add reply-to: header for users@ announcement.
-
- - Patch/verify the announcement URL in release notes, re-publish.
-
- - Announce on reddit and other places (see doc/ann/).
-
- - Commit and tag private/ and style/
-
- git tag -a 0.7.0 -m "Tag version 0.7.0" && git push --follow-tags
-
-- Tag & Commit hello
-
- cd hello
- ./foreach-git stat
-
- - commit hello/repository/ (see commit history for procedure)
-
- git tag -a build2-0.7.0 -m "Tag for build2 version 0.7.0" && git push --follow-tags
-
-
-- Tag & Commit
-
- - commit & push cppget.org/repository/ (see commit history for procedure;
- commit big chunks first, then reorder with rebase).
-
- - Tag and push all released packages, some by hand, some with tag scripts
- (see etc/stage for list but also packages added ad hoc with etc/stage-pkg).
-
- @@ Use vX.Y.Z tag for packages that are bpkg/bdep-ready (i.e., have
- suitable repositories.manifest if required).
-
- cd <proj>
-
- # X.Y.Z
- v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \
- git tag -a $v -m "Tag version $v" && git push --follow-tags
-
- # vX.Y.Z
- v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo v$v; read; \
- git tag -a v$v -m "Tag version $v" && git push --follow-tags
-
- For build2 projects, review tag_modules in etc/git/modules, then:
-
- ./tag.sh 0.7.0
- ./push.sh
-
-- Increment versions and open master for business
-
- - Change to next development versions for all released packages, some by
- hand, some with version scripts (see etc/stage for list + msvc-linux).
-
- cd <proj>
-
- ~ edit manifest
-
- v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \
- git ci -a -m "Bump version to $v, master is open for business" && git push
-
- - In build2-toolchain, update all submodules (including libodb*, etc),
- change BUILD2_REPO to staging.
-
- - Regenerate odb & cli docs everywhere (packages and build2-toolchain).
-
- New sequence:
-
- ~ rebuild ODB compiler
-
- @@ bdep sync -fura
- @@ b hxx{version} # first lib*, then projects, then odb.sh
-
- - Update/test local (see early steps of this list).
-
- - Make symlinks for new version in baseutils (for both baseutils and mingw,
- the idea is that we will start with those and maybe upgrade during
- development).
-
- - Restage
-
- rm -r staging/0/*
- rm -r staging/repository/1/*/
-
- etc/stage -b
-
- - Make sure bbot builds of the new development snapshot are successful.
-
-- Commit etc/
-
- - Restore changes to etc/git/modules
-
- - Change all '+ ' back to '- '.
-
- - Commit and tag
-
- git tag -a 0.7.0 -m "Tag version 0.7.0" && git push --follow-tags
-
-- Snapshot buildos subvolumes as buildos-0.7.0
-
- btrfs subvolume snapshot buildos-3 buildos-3-0.7.0
- btrfs subvolume snapshot buildos-6 buildos-6-0.7.0
diff --git a/rep-test b/rep-test
deleted file mode 100755
index 09541b6..0000000
--- a/rep-test
+++ /dev/null
@@ -1,179 +0,0 @@
-#! /usr/bin/env bash
-
-# Test repository.
-#
-# Usage: update [options] <dir>
-#
-# First, the script determines the list of repositories/sections. If <dir>
-# contains the repositories.manifest file, then it is the only repository to
-# be tested. Otherwise, every first-level subdirectory of <dir> that doesn't
-# start with '.' and contains the repositories.manifest file is to be tested.
-#
-# Then, it makes sure that every package in every repository can be built
-# in a clean configuration.
-#
-# -n
-# Only test new packages. For this to work, <dir> should be (part of) a
-# git repository. Untracked (and changed) files are considered new.
-#
-# -e <subdir>
-# Exclude the specified sub-directory. Currently only one directory can
-# be excluded.
-#
-# -t <toolchain>
-# Specify the build2 toolchain install/stage directory. The script will
-# use <toolchain>/bin/b and <toolchain>/bin/bpkd instead of just b and
-# bpkg.
-#
-# -c <option>|<module>|<var>
-# Specify additional options, modules, or configuration variables to pass
-# to the bpkg create command. For example:
-#
-# -c cxx -c config.cxx.loptions=-L/usr/local/lib
-#
-usage="usage: $0 [options] <dir>"
-
-trap 'exit 1' ERR
-set -o errtrace # Trap in functions.
-
-function info () { echo "$*" 1>&2; }
-function error () { info "$*"; exit 1; }
-
-dir=
-cfg=/tmp/test-cfg
-toolchain=
-exclude=
-co=() # Use a bash array to handle arguments with spaces (say "-Ifoo -Ibar").
-new=n
-
-while [ $# -gt 0 ]; do
- case $1 in
- -e)
- shift
- exclude=${1%/}
- shift
- ;;
- -t)
- shift
- toolchain=${1%/}
- shift
- ;;
- -c)
- shift
- co=("${co[@]}" "$1")
- shift
- ;;
- -n)
- new=y
- shift
- ;;
- *)
- dir=${1%/}
- shift
- break
- ;;
- esac
-done
-
-if [ -z "$dir" ]; then
- error $usage
-fi
-
-if [ -z "$toolchain" ]; then
- b="b"
- bpkg="bpkg"
-else
- b="$toolchain/bin/b"
- if [ ! -x $b ]; then
- error "$b does not exist or is not executable"
- fi
- bo="--build $b"
-
- bpkg="$toolchain/bin/bpkg"
- if [ ! -x $bpkg ]; then
- error "$bpkg does not exist or is not executable"
- fi
-fi
-
-if [ "$new" = "y" ]; then
- new=( `git -C $dir status --untracked-files=all --porcelain . | \
-sed -e 's%^\(??\| M\|M \) .*/\(.*\)$%\2%'` )
- if [ ${#new[@]} -eq 0 ]; then
- info "no new packages in $dir"
- exit 0
- fi
- info "testing only ${new[@]}"
-else
- new=()
-fi
-
-function check_new () # <pkg> <ver>
-{
- if [ ${#new[@]} -eq 0 ]; then
- return 0
- fi
-
- for f in ${new[@]}; do
- if [[ $f == $1-$2.?* ]]; then
- return 0
- fi
- done
-
- return 1
-}
-
-# Determine the list of repositories.
-#
-rep=
-
-if [ -f "$dir/repositories.manifest" ]; then
- rep=$dir
-else
- sub=`find $dir -mindepth 1 -maxdepth 1 -type d -name '[^.]*'`
- if [ -z "$sub" ]; then
- error "no subdirectories in $dir/"
- fi
-
- for d in $sub; do
- if [ `basename $d` = "$exclude" ]; then
- info "skipping excluded $d/"
- continue
- fi
- if [ -f "$d/repositories.manifest" ]; then
- rep+=" $d"
- else
- info "no repositories.manifest file in $d/, skipping"
- fi
- done
-
- if [ -z "$rep" ]; then
- error "no repositories in $dir/"
- fi
-fi
-
-# Test repositories.
-#
-for r in $rep; do
- info "testing $r"
-
- IFS=$'\n'
- pkg=( `$bpkg rep-info --packages $r` )
- IFS=$' \t\n'
-
- for p in "${pkg[@]}"; do
- nv=( $p )
- n=${nv[0]}
- v=${nv[1]}
-
- if ! check_new $n $v; then
- continue
- fi
-
- info "testing $n $v"
-
- $bpkg $bo create -d $cfg --wipe "${co[@]}"
- $bpkg add -d $cfg $r
- $bpkg fetch -d $cfg
- $bpkg $bo build -d $cfg --yes $n/$v
- done
-done
diff --git a/review b/review
index 0f6c739..8a46b4e 100755
--- a/review
+++ b/review
@@ -6,7 +6,7 @@
#
usage="usage: $0"
-modules="libbutl build2 libbpkg bpkg bdep brep libbbot bbot libstd-modules build2-toolchain"
+modules="libbutl libbutl.bash build2 libbpkg bpkg bpkg-rep bdep brep libbbot bbot libstd-modules build2-toolchain"
owd=`pwd`
trap "{ cd $owd; exit 1; }" ERR
diff --git a/stage b/stage
index 4c3bc6b..19944f5 100755
--- a/stage
+++ b/stage
@@ -1,6 +1,6 @@
#! /usr/bin/env bash
-# Stage or queue build2 packages and distributions.
+# Stage or queue build2 packages and the toolchain distribution.
#
# -b
# Packages baseutils/mingw.
@@ -9,11 +9,11 @@
# -s
# Only prepare the install scripts and regenerate toolchain.sha256.
# -q
-# Put packages into the queue instead of staging. Used to update for
-# queue.
+# Put packages and the distribution into the queue instead of staging.
# -p
-# Regenerate build2-toolchain into queue's 0/ and update the checksums file
-# but don't do anything else. Used to update for public.
+# Regenerate and publish distribution. Specifically, move baseutils from
+# queue, regenerate build2-toolchain, install scripts, and the checksum
+# file, and upload everything to public. Also disable the queue toolchain.
#
usage="usage: etc/stage"
@@ -26,6 +26,12 @@ set -o errtrace # Trap in functions.
function info () { echo "$*" 1>&2; }
function error () { info "$*"; exit 1; }
+# Make sure the build2 tools are runnable.
+#
+b --version >/dev/null
+bpkg --version >/dev/null
+bdep --version >/dev/null
+
v="$(sed -n -re 's/^version: ([^.]+\.[^.]+\.[^-]+(-[ab]\.[^.+]+)?).*$/\1/p' build2-toolchain/manifest)"
vf="$(echo $v | sed -re 's/^([^.]*)\.([^.]*)\.([^-]*).*$/\1.\2.\3/')" # Final.
@@ -41,6 +47,8 @@ script_only=
dist_src=staging/0
dist_dst=stage.build2.org:/var/www/stage.build2.org/public/0
+rsync_del="--delete-after"
+
while [ $# -gt 0 ]; do
case $1 in
-b)
@@ -48,6 +56,7 @@ while [ $# -gt 0 ]; do
shift
;;
-o)
+ base=true
base_only=true
shift
;;
@@ -55,16 +64,22 @@ while [ $# -gt 0 ]; do
script_only=true
shift
;;
- -q)
- mode=-q
- dist_src=cppget.org/0
+ -q | -p)
+ mode="$1"
+ dist_src=cppget.org/queue/0
dist_dst=build2.org:/var/www/download.build2.org/public/queue
- shift
- ;;
- -p)
- mode=-p
- dist_src=cppget.org/0
- dist_dst=example.org/not-used
+
+ if [ "$mode" = "-p" ]; then
+
+ queue_dist_src="$dist_src"
+ queue_dist_dst="$dist_dst"
+
+ dist_src=cppget.org/public/0
+ dist_dst=build2.org:/var/www/download.build2.org/public
+
+ rsync_del= # Do not delete anything existing.
+ fi
+
shift
;;
*)
@@ -113,24 +128,42 @@ if [ "$base" = true ]; then
--min-s ./pkpost-mingw-mingw
cd ../..
+fi
+
+# Copy baseutils if requested.
+#
+if [ "$base" = true -o "$mode" = "-p" ]; then
+
+ if [ "$base" = true ]; then
+ d=/tmp
+ else
+ d="$queue_dist_src/$v"
+ fi
- mv "/tmp/build2-baseutils-$v-i686-windows.zip" "$dist_src/$v/"
- mv "/tmp/build2-baseutils-$v-i686-windows.zip.sha256" "$dist_src/$v/"
+ mv "$d/build2-baseutils-$v-i686-windows.zip" "$dist_src/$v/"
+ mv "$d/build2-baseutils-$v-i686-windows.zip.sha256" "$dist_src/$v/"
- mv "/tmp/build2-baseutils-$v-x86_64-windows.zip" "$dist_src/$v/"
- mv "/tmp/build2-baseutils-$v-x86_64-windows.zip.sha256" "$dist_src/$v/"
+ mv "$d/build2-baseutils-$v-x86_64-windows.zip" "$dist_src/$v/"
+ mv "$d/build2-baseutils-$v-x86_64-windows.zip.sha256" "$dist_src/$v/"
- mv "/tmp/build2-mingw-$v-i686-windows.tar.xz" "$dist_src/$v/"
- mv "/tmp/build2-mingw-$v-i686-windows.tar.xz.sha256" "$dist_src/$v/"
+ mv "$d/build2-mingw-$v-i686-windows.tar.xz" "$dist_src/$v/"
+ mv "$d/build2-mingw-$v-i686-windows.tar.xz.sha256" "$dist_src/$v/"
- mv "/tmp/build2-mingw-$v-x86_64-windows.tar.xz" "$dist_src/$v/"
- mv "/tmp/build2-mingw-$v-x86_64-windows.tar.xz.sha256" "$dist_src/$v/"
+ mv "$d/build2-mingw-$v-x86_64-windows.tar.xz" "$dist_src/$v/"
+ mv "$d/build2-mingw-$v-x86_64-windows.tar.xz.sha256" "$dist_src/$v/"
fi
if [ "$base_only" = true ]; then
exit 0
fi
+# Clean and disable queue toolchain.
+#
+if [ "$mode" = "-p" ]; then
+ rm -rf "$queue_dist_src/$v"
+ echo 'disabled' >"$queue_dist_src/toolchain.sha256"
+fi
+
##
##if false; then
@@ -183,19 +216,32 @@ for f in "${scripts[@]}"; do
checksum_line "$f"
done
-if [ "$mode" = "-p" ]; then
- exit 0
-fi
-
# Sync the distribution.
#
# Note that we do the distribution first so that the toolchain changes are
# visible before package changes. This is relied upon by the Build OS monitor.
#
-info "Press Enter to start distribution upload"
-read
-rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \
- $rsync_ops "$dist_src/" "$dist_dst/"
+function sync_dist () # <dist-src> <dist-dst> [<rsync-options>]
+{
+ local s="$1"
+ local d="$2"
+
+ shift
+ shift
+
+ info "Press Enter to start distribution upload to $d"
+ read
+ rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs \
+ $rsync_ops "$@" "$s/" "$d/"
+}
+
+sync_dist "$dist_src" "$dist_dst" $rsync_del
+
+# Also sync (disable) queue.
+#
+if [ "$mode" = "-p" ]; then
+ sync_dist "$queue_dist_src" "$queue_dist_dst" --delete-after
+fi
##fi
##
@@ -204,12 +250,16 @@ if [ "$script_only" = true ]; then
exit 0
fi
+if [ "$mode" = "-p" ]; then
+ exit 0
+fi
+
# Distribute packages, regenerate and sync the repository.
#
# These are just for testing (though libodb-mysql needs libmysqlclient).
#
-etc/stage-pkg -c $mode -d vcpkg packaging/vcpkg/vcpkg
+#etc/stage-pkg -c $mode -d vcpkg packaging/vcpkg/vcpkg
etc/stage-pkg -c $mode -d mariadb packaging/mariadb/libmariadb
etc/stage-pkg -c $mode -d mysql packaging/mysql/libmysqlclient
@@ -233,7 +283,7 @@ etc/stage-pkg -c $mode -d libstudxml ../studxml/libstudxml
etc/stage-pkg -c $mode -d hello hello/libhello-1.0
-# For now we also stage libodb-mysql.
+# For testing we also stage other libodb-*.
#
etc/stage-pkg -c $mode -d odb \
../odb/odb \
@@ -242,7 +292,10 @@ etc/stage-pkg -c $mode -d odb \
../odb/builds/default/libodb-pgsql \
../odb/builds/default/libodb-mysql \
../odb/builds/default/libodb-oracle \
- ../odb/builds/default/libodb-mssql
+ ../odb/builds/default/libodb-mssql \
+ ../odb/builds/default/libodb-boost \
+ ../odb/builds/default/libodb-qt
+
etc/stage-pkg -c $mode build2 \
libbutl \
@@ -250,8 +303,12 @@ etc/stage-pkg -c $mode build2 \
build2 \
libbpkg \
bpkg \
+ bpkg-rep \
bdep \
libbbot \
brep \
bbot \
- libstd-modules
+ openssl-agent
+
+# msvc-linux
+# libstd-modules
diff --git a/stage-pkg b/stage-pkg
index 1a77e1f..6930fd9 100755
--- a/stage-pkg
+++ b/stage-pkg
@@ -12,7 +12,7 @@
# Distribute and regenerating only without publishing the repository.
#
# -q
-# Publish packages into the queue instead of staging.
+# Publish packages into the queue instead of staging. Implies -d.
#
usage="usage: etc/stage-pkg [<options>] <group> <dir>..."
@@ -25,6 +25,12 @@ set -o errtrace # Trap in functions.
function info () { echo "$*" 1>&2; }
function error () { info "$*"; exit 1; }
+# Make sure the build2 tools are runnable.
+#
+b --version >/dev/null
+bpkg --version >/dev/null
+bdep --version >/dev/null
+
repo_name="STAGE.BUILD2.ORG"
repo_dir="staging/repository/1"
repo_host1="stage.build2.org:/var/bpkg/1"
@@ -51,10 +57,11 @@ while [ $# -gt 0 ]; do
shift
;;
-q)
- repo_name="CPPGET.ORG/QUEUE"
- repo_dir="cppget.org/repository/1/queue"
- repo_host1="cppget.org:/var/bpkg/1/queue"
- repo_host2="queue.cppget.org:/var/bpkg/1/queue"
+ #repo_name="CPPGET.ORG/QUEUE"
+ repo_dir="cppget.org/queue/1/alpha"
+ #repo_host1="cppget.org:/var/bpkg/1/queue"
+ #repo_host2="queue.cppget.org:/var/bpkg/1/queue"
+ dist_only=true
shift
;;
*)
diff --git a/upgrade b/upgrade
new file mode 100755
index 0000000..e2b0456
--- /dev/null
+++ b/upgrade
@@ -0,0 +1,131 @@
+#! /usr/bin/env bash
+
+# Upgrade remote packages in a bdep-managed build2 toolchain build,
+# essentially as if by executing:
+#
+# bdep sync -fur && b
+#
+# In each project of the toolchain.
+#
+# Note that we can't just do that because as soon as we drop some dependency
+# package, the tools (b, bppg, etc) become non-runnable (missing shared
+# libraries, etc).
+#
+# Note also that this script expects the build system (build2) and the rest of
+# the toolchain (bpkg, bdep, etc) to be built in separate build configuration
+# (for example, tsan for build2 and asan for the rest).
+#
+# Finally, this script only upgrades the default configurations. To upgrade
+# the rest you can use the normal way, for example:
+#
+# bdep sync -fura
+#
+# If the -c option is specified, then the second configuration is upgraded
+# and configured but is not updated. This can be done at a later stage by
+# running the build system in the configuration directory:
+#
+# BDEP_SYNC=0 b
+#
+trap "exit 1" ERR
+set -o errtrace # Trap in functions.
+
+function info () { echo "$*" 1>&2; }
+function error () { info "$*"; exit 1; }
+
+# Make sure the build2 tools are runnable.
+#
+b --version >/dev/null
+bpkg --version >/dev/null
+bdep --version >/dev/null
+
+configure_only=
+
+while [ $# -gt 0 ]; do
+ case $1 in
+ -c)
+ configure_only=--configure-only
+ shift
+ ;;
+ *)
+ error "unexpected $1"
+ ;;
+ esac
+done
+
+# Get the configuration directories.
+#
+bcfg="$(b info: build2/ | sed -n -re 's/^out_root: (.+)$/\1/p')"
+pcfg="$(b info: bpkg/ | sed -n -re 's/^out_root: (.+)$/\1/p')"
+
+if [ -z "$bcfg" -o -z "$pcfg" ]; then
+ error "unable to determine build configuration directories"
+fi
+
+bcfg="$(dirname "$bcfg")"
+pcfg="$(dirname "$pcfg")"
+
+if [ "$bcfg" = "$pcfg" ]; then
+ error "build2 and bpkg build configuration directories are the same"
+fi
+
+# The plan is as follows:
+#
+# 0. First, make backup copies of both configurations. If something goes wrong,
+# it's likely the tools will be left in a non-runnable state.
+#
+# 1. Next, install the build system on the side and then upgrade the build
+# system configuration using that plus the existing bpkg/bdep.
+#
+# 2. Finally, upgrade the rest. Note that we only have one shot at this since
+# we are running bpkg that is being upgraded.
+#
+
+# Step 0.
+#
+if test -e "$bcfg.bak"; then
+ error "$bcfg.bak already exist"
+fi
+
+if test -e "$pcfg.bak"; then
+ error "$pcfg.bak already exist"
+fi
+
+set -x
+
+cp -rp "$bcfg" "$bcfg.bak"
+cp -rp "$pcfg" "$pcfg.bak"
+
+# Step 1.
+#
+bpkg install -d "$bcfg" \
+ config.install.root=/tmp/build2-install \
+ config.bin.rpath=/tmp/build2-install/lib \
+ build2
+
+/tmp/build2-install/bin/b --version >/dev/null
+
+bpkg fetch -d "$bcfg"
+BDEP_SYNC=0 bpkg build -d "$bcfg" --build /tmp/build2-install/bin/b --keep-out -ur
+
+b --version >/dev/null
+
+b build2/ # Should be a noop.
+
+rm -rf /tmp/build2-install
+
+# Step 2.
+#
+bpkg fetch -d "$pcfg"
+BDEP_SYNC=0 bpkg build -d "$pcfg" --keep-out -ur "$configure_only"
+
+if [ -z "$configure_only" ]; then
+
+ bpkg --version >/dev/null
+ bdep --version >/dev/null
+
+ b bpkg/ bdep/ # Should be a noop.
+
+fi
+
+rm -rf "$bcfg.bak"
+rm -rf "$pcfg.bak"
diff --git a/version b/version
index 970402e..6bd036b 100755
--- a/version
+++ b/version
@@ -13,22 +13,21 @@ usage="usage: etc/version"
build2_min=0.8.0-
bpkg_min=0.8.0-
-ver=0.8.0-a.0.z
-#ver=0.7.0
-mods=(libstd-modules libbutl libbutl.bash libbpkg libbbot)
+#ver=0.10.0-a.0.z
+ver=0.9.0
+mods=(libstd-modules libbutl libbutl.bash libbpkg libbbot bpkg)
-odb_ver=2.5.0-b.8.z
-#odb_ver=2.5.0-b.7
+#odb_ver=2.5.0-b.12.z
+odb_ver=2.5.0-b.11
odb_mods=(libodb libodb-sqlite libodb-pgsql)
-xml_ver=1.1.0-b.3.z
-#xml_ver=1.1.0-b.3
+#xml_ver=1.1.0-b.6.z
+xml_ver=1.1.0-b.5
xml_mods=(libstudxml)
-# @@ NOTE: libpkgconf manually.
-
-all=(libstd-modules libbutl libbutl.bash build2 libbpkg bpkg bdep libbbot bbot brep build2-toolchain msvc-linux buildos)
+# @@ NOTE: libpkgconf version constrain is updated manually.
+all=(libstd-modules libbutl libbutl.bash build2 libbpkg bpkg bpkg-rep bdep libbbot bbot brep build2-toolchain msvc-linux openssl-agent buildos)
# In manifests for alpha/beta we use an exact match. For snapshots -- snapshot
# range. Otherwise an X.Y.* range.