0.4.0 release updates0.4.0

1 files changed, 206 insertions, 98 deletions

diff --git a/release.txt b/release.txt
index cd133a5..ad06109 100644
--- a/release.txt
+++ b/release.txt
@@ -1,14 +1,12 @@
 TODO
 ====
 
-@@ Local Clang test is broken and disabled, using FreeBSD for Clang testing
+@@ No upgrade testing (see old test-upgrade script).
 
-Notes
-=====
-
-* Be very careful with submodules if fixing anything, remember to update
-  build2-toolchain.
+@@ Not saving in previous toolchains (see old install script).
 
+@@ Using FreeBSD for Clang/libc++ testing, both 3.4 (earliest we support)
+   and 3.7.
 
 Setup
 =====
@@ -21,44 +19,214 @@ Setup
   For Mac OS:
     echo "boris ALL=(ALL) NOPASSWD:ALL" >>/etc/sudoers
 
-- Boot cppget1, freebsd VMs, make sure data/time is correct.
+- Boot freebsd VM, make sure data/time is correct.
+
+Notes
+=====
+
+* Be very careful with submodules if fixing anything, remember to update
+  build2-toolchain.
+
+* Be very carefult with regenerating documentation, ODB files, both in
+  individual projects and build2-toolchain.
+
+Stage
+=====
+
+- To stage build2
+
+  ~ make sure build2-toolchain is ready (submodules, doc/odb regen)
+  ~ update BUILD2_REPO URL in build2-toolchain/build* scripts if needed
+
+  To stage to stage.build2.org:
+
+    ~ to exclude baseutils/mingw, add -p
+
+    etc/stage
+
+  To stage to cppget.org/queue:
+
+    ~ note: assuming brep running on queue can handle it
+
+    etc/stage -q
+
+- To stage hello
+
+  cd hello
+  ./stat
+  ./stage
+
+  To run intro on staged:
+
+    etc/intro -s https://build2.org/pkg/1/stage
+
+Test :
+
+ + Local with cli disables
+
+ + Local with Intel icc
+   ~ fails with ICE on build2
+
+ + Local with gcc 4.8
 
-+ Copy prerequisites into build2-X.Y/, generate sha256 sums in BINARY (-b)
+ + Local brep build:
 
-+ Build new prerequisites (if any) on local, freebsd (install to /usr/local).
+   bpkg create -d brep cc config.cc.poptions="-I$(apxs -q includedir)"
+   cd brep
+   bpkg add https://stage.build2.org/1
+   bpkg fetch
+   bpkg build brep ?sys:libapr1 ?sys:libpq
 
-+ If new preprequisites/version, need to update manifests.
+Cross:
+
+ + Test VC cross-build
+
+ + Test MinGW cross-build (build2 build fails on GC 4.8.2, using ODB cross)
+
+
+Bootstrap (with upgrade procedures):
+
+ + CentOS GCC 4.8
+
+ + FreeBSD 10 Clang 3.4 (clang++)
+
+ + FreeBSD 10 Clang 3.7 (clang++37) with -W -Wall (mod build.sh) + brep:
+
+   bpkg create -d brep cc config.cxx=clang++37 \
+     config.cc.poptions="-I$(apxs -q includedir)"
+   cd brep
+   bpkg add https://stage.build2.org/1
+   bpkg fetch
+   bpkg build brep ?sys:libapr1 ?sys:libpq
+
+ + Windows7-64 VC14u2
+
+ + WindowsXP-32 MinGW (build2-mingw)
+
+ + Windows MSYS2
+
+ + Mac OS Xcode/Clang (and g++ alias)
+
+Upgrade brep on stage
 
 Prelude
 =======
 
-+ Remove etc/ from modules in etc/git/modules.
++ Remove etc/, build2-toolchain from modules in etc/git/modules.
 
 + Review '@@' items [note: etc, private excluded]
 
   etc/review | less -R
 
-+ Need to make sure Intro is still accurate wrt output, etc. Use -s switch
-  to etc/intro to compare:
++ Make sure Intro is still accurate wrt output, etc. Use -s -p switches to
+  etc/intro to compare (after running hello/stage):
 
   etc/intro -s `pwd`/hello/repository/1/
 
++ If new preprequisites/version, need to update manifests. See
+  private/build2-projects.txt to review.
+
 + Increment version to final.
 
-  * Need to update versions/dates for man generation (cli.sh in build2,
-    bpkg, brep, and built2-toolchain).
+  ~ update and use etc/version script
 
-  * Review manifests for any other changes (new entries/files, etc).
+  ~ update versions/dates for man generation (cli.sh in build2, bpkg, brep,
+    and built2-toolchain)
+
++ Update BUILD2_REPO URL in build2-toolchain/build* scripts to alpha, also
+  grep docs for 'stage.' in case got copied.
 
 + Close schema versions, review schema changlog difference from previous
   release (tag) for any data migration that may be required (@@ would also
   need to test this; see upgrade test).
 
-+ Need to update submodules? Remember to push first.
++ Update NEWS files.
+
++ Need to update submodules? (remember to push first)
+
++ Need to regenerate ODB files, CLI documentation, in build2-toolchin?
+
+Procession
+==========
+
++ Stage final packages (remove -p if need baseutils/mingw)
+
+  etc/stage -p
+
+- Upgrade brep on lists.build2, lists.cppget from stage
+
+  ~ assumming it can handle old repositories (should normally be the case)
+
+  ~ backup config if changed
+
+    cd private/
+    scp -r lists.build2.org:/home/brep/config lists.build2.org/home/brep/
+    scp -r lists.cppget.org:/home/brep/config lists.cppget.org/home/brep/
+
+  ~ note: on cppget we have queue:
+
+    install/bin/brep-migrate -n brep_queue
+    install/bin/brep-load -n brep_queue config/brep-queue-load.conf
+
++ Queue final packages
+
+  etc/stage -p -q
+
++ Smoke-test any last-minute changes
 
-+ Need to regenerate ODB files, CLI documentation, in build2-toolchin!
++ Move packages from queue to alpha, cleanup obsolete, regenerate.
 
-+ Update everything:
+  cd cppget.org
+  git -C repository status
+  ./update
+
++ Publish to production (build2.org/cppget.org)
+
+  etc/publish
+
++ Smoke-test
+
++ Publish hello repository
+
+  cd hello
+  ./stage -p
+
+  Test intro steps:
+
+  etc/intro -s https://build2.org/pkg/1/hello
+
+- Commit and push git repositories:
+
++  ~ cppget.org (commit individually and push)
+
+   ~ hello project and respository (add & commit individual packages)
+
+    cd hello
+    ./foreach-git stat
+
+    ./foreach-git -x libhello-1.0 tag -a build2-X.Y -m "Tag for build2 version X.Y"
+    git -C libhello-1.0 tag -a 1.0/build2-X.Y -m "Tag for build2 version X.Y"
+    ./foreach-git push --tags
+
++ Tag (review tag_modules) and push.
+
+  etc/tag
+  ./push.sh
+
+- Tag and increment version for each published package (see
+  private/build2-projects.txt)
+
+=============================================================================
+
+@@ When copying to production need to make sure we don't override existing
+   versions of prerequisite packages that haven't changed (but which we still
+   stage/queue).
+
+=============================================================================
+
+@@ Move up.
+
+- Update everything:
 
   b libbutl/ build2/ libbpkg/ bpkg/ brep/ build2-toolchain-default/
 
@@ -74,24 +242,6 @@ Prelude
     ./test.sh -v --remote --fetch fetch --fetch-option --no-verify-peer \
       --tar bsdtar --sha256 sha256-freebsd
 
-  * Update cppget repository (existing packages, not publishing yet):
-
-    @@ What is the purpose of this step? It will mess up upgrade test
-    below.
-
-    -t /home/boris/work/build2/etc1/install/0.2.0 -e queue
-
-    cd cppget.org
-    git -C repository status
-    ./update repository/1/
-
-    This might not work in the early stage (backwards-incompatible)
-
-    ./test -c cxx \
-    -c config.cxx.loptions=-L/usr/local/lib \
-    -c config.cxx.poptions="-I/usr/include/apache2 -I/usr/include/apr-1.0" \
-      repository/1/
-
   * Test dev integration of brep/ on hello and cppget repos
 
     First check if Apache2 configuration needs updating (INSTALL-DEV). Then
@@ -114,54 +264,33 @@ Prelude
 Procession
 ==========
 
-- Generate distribution (use -t for toolchain-only, without brep). The
+X Generate distribution (use -t for toolchain-only, without brep). The
   result is placed into the queue.
 
   etc/dist 2>&1 | tee dist.log
 
-- @@ Would be nice to test packages for missing files, etc., before upgrade.
-
-- Test upgrade from previous version (replace Y in 0.Y.0).
+X Test upgrade from previous version (replace Y in 0.Y.0).
 
   etc/test-upgrade -c cxx \
     -c config.cxx.loptions=-L/usr/local/lib \
     -c config.cxx.poptions="-I/usr/include/apache2 -I/usr/include/apr-1.0" \
     /home/boris/work/build2/etc1/install/0.Y.0
 
-- Update and test hello repository
+X Update and test hello repository
 
   cd hello
-  for d in hello libhello libprint libformat repository; do \
-    git -C $d status; done
-  ./release -a
+  ./foreach-git status
+  ./stage -p
 
-- Test Intro steps on the local hello repository
+X Test Intro steps on the local hello repository
 
   etc/intro `pwd`/hello/repository/1/
 
-- Publish hello repository, commit but don't push.
-
-  [We need to publish now because etc/test uses it.]
-
-  @@ Ideally we would want to stage it.
-
-  ./release -p
-
-- Test Intro steps on the remote hello repository
-
-  etc/intro https://build2.org/pkg/1/hello/
-
-- Stage queue to cppget.org
+X Stage queue to cppget.org
 
   etc/stage --dry-run
   etc/stage
 
-- Regenerate other cppget.org sections with the new toolchain
-
-  cd cppget.org
-  git -C repository status
-  ./update -e queue repository/1/
-
 - Determine the earliest supported backwards-compatible toolchain (see
   requires: in manifests), update etc/test
 
@@ -170,21 +299,6 @@ Procession
   etc/test 2>&1 | tee test.log
   grep -i warning test.log
 
-- Upgrade brep on build2.org (via package/queue upgrade if possible),
-  verify works.
-
-  Backup configs if changed:
-
-  cd private/
-  scp -r lists.build2.org:/home/brep/config lists.build2.org/home/brep/
-
-- Move packages from queue to alpha, regenerate repositories.
-
-  ./update repository/1/
-
-- Publish to production (build2.org/cppget.org).
-
-  etc/publish
 
 - Test production and save a copy of toolchain in etc1/install/X.Y.Z
 
@@ -205,51 +319,45 @@ Procession
   cd private/
   scp -r lists.cppget.org:/home/brep/config lists.cppget.org/home/brep/
 
-- Tag (review tag_modules)
-
-  etc/tag
-  ./push.sh
-
-- Commit and push cppget.org, hello git repositories
-
 Release
 =======
 
-- Update style/ submodule in private/
++ Update style/ submodule in private/
 
-- Write release notes, use placeholder for announcement URL. Add link to
++ Write release notes, use placeholder for announcement URL. Add link from
   the doc page.
 
 - Any new FAQ entries or any other updates (main page, etc)?
 
-- Update download page.
++ Update download page.
 
-- Regenerate (./cli.sh) all hosts.
++ Regenerate (./cli.sh) all hosts.
 
-- Test local, publish, test production:
++ Test local, publish, test production:
 
   cd private/
   ./publish --dry-run
   ./publish
 
-- Write and send announcements
++ Write and send announcements
 
   Remember upgrade instructions.
 
   cat `ls -1 *.sha256`
 
-- Patch the announcement URL in release notes, re-publish.
++ Patch the announcement URL in release notes, re-publish.
 
 - Announce on reddit and other places (doc/ann).
 
 Postlude
 ========
 
-- Need to regenerate/republish hello/ repository?
-
 - Increment and open schema versions. Maybe not. Maybe we should only do that
   when there is a model change.
 
-- Increment project versions to alpha1, open master for business
++ Increment project versions to alpha1, change BUILD2_REPO to staging,
+  open master for business.
+
+- Commit changes to etc/, add it back to etc/git/modules, and tag:
 
-- Commit changes to etc/, add it back to etc/git/modules.
+  git tag -a X.Y.Z -m "Tag version X.Y.Z"