diff options
| -rwxr-xr-x | doc/cli.sh | 7 | ||||
| -rw-r--r-- | doc/intro2.cli | 1158 |
2 files changed, 1164 insertions, 1 deletions
@@ -36,6 +36,9 @@ function gen () # <name> --html-epilogue-file doc-epilogue.xhtml \ --link-regex '%b([-.].+)%../../build2/doc/b$1%' \ --link-regex '%bpkg([-.].+)%../../bpkg/doc/bpkg$1%' \ +--link-regex '%bdep([-.].+)%../../bdep/doc/bdep$1%' \ +--link-regex '%b(#.+)?%../../build2/doc/build2-build-system-manual.xhtml$1%' \ +--link-regex '%bpkg(#.+)?%../../bpkg/doc/build2-package-manager-manual.xhtml$1%' \ --output-prefix build2-toolchain- "${@}" $n.cli html2ps -f doc.html2ps:a4.html2ps -o build2-toolchain-$n-a4.ps build2-toolchain-$n.xhtml @@ -45,10 +48,12 @@ html2ps -f doc.html2ps:letter.html2ps -o build2-toolchain-$n-letter.ps build2-to ps2pdf14 -sPAPERSIZE=letter -dOptimize=true -dEmbedAllFonts=true build2-toolchain-$n-letter.ps build2-toolchain-$n-letter.pdf } +gen intro2 +gen intro + # Auto-heading doesn't work since it is broken into multiple doc strings. # gen install --html-heading-map 2=h2 -gen intro # Generate INSTALL/BOOTSTRAP/UPGRADE file in ../ # diff --git a/doc/intro2.cli b/doc/intro2.cli new file mode 100644 index 0000000..1d62ee6 --- /dev/null +++ b/doc/intro2.cli @@ -0,0 +1,1158 @@ +// file : doc/intro2.cli +// copyright : Copyright (c) 2014-2017 Code Synthesis Ltd +// license : MIT; see accompanying LICENSE file + +"\name=build2-toolchain-intro" +"\subject=toolchain" +"\title=Toolchain Introduction" + +// TODO +// +// @@ refs to further docs +// +// STYLE +// +// @@ section boundary page breaks (<hr class="page-break"/>) +// @@ when printed, code background is gone, but spaces still there +// +// PDF +// +// @@ tree output is garbled +// @@ Could we use a nicer font, seeing that we embed them? +// + +// NOTES +// +// - Maximum <pre> line is 70 characters. +// + +" +\h1#tldr|TL;DR| + +\ +$ git clone ssh://example.org/hello.git +$ cd hello + +$ bdep init --create ../hello-gcc cc config.cxx=g++ +$ b +<compile> +$ ./hello + +$ edit repositories.manifest # add https://example.org/libhello.git +$ edit manifest # add 'depends: libhello ^ 1.0.0' +$ edit buildfile # import libhello +$ edit hello.cxx # use libhello +$ b +<sync> +<compile> + +$ bdep fetch # refresh available versions +$ bdep status # review available versions +<show new version> + +$ bdep sync libhello # upgrade to latest +<...> + +$ bdep sync libhello/1.1.0 # downgrade to X.Y.Z +<...> +\ + +\h1#tour|A Tour of The \c{build2} Toolchain| + +The aim of this section is to give you a quick tour of the \c{build2} +toolchain with minimal explanation of the underlying concepts with the +subsequent sections going into more detail. + +All the examples in this document include the relevant command output so that +you don't have to install the toolchain and run the commands in order to +follow alone. If at the end you find \c{build2} appealing and would like to +try the examples for yourself, you can jump straight to +\l{build2-toolchain-install.xhtml The \c{build2} Toolchain Installation and +Upgrade}. + +The question we will try to answer in this section can be summarized as: + +\ +$ git clone .../hello.git && now-what? +\ + +That is, we clone an existing C++ project or would like to create a new one +and then start hacking on it. We want to spend as little time and energy as +possible on the initial and ongoing infrastructure maintenance: setting up +build configurations, managing dependencies, continuous integration and +testing, etc. Or, as one C++ user aptly put it, \"\i{All I want to do is +program.}\" + +\h#tour-hello|Hello, World| + +Let's see what programming with \c{build2} feels like by starting with a +customary \i{\"Hello, World!\"} program: + +\ +$ bdep new -t exe -l c++ hello +\ + +The \l{bdep-new(1)} command creates a \i{canonical} \c{build2} project. In +our case it is an executable implemented in C++. + +\N|To create a library, pass \c{-t\ lib}. By default \c{new} also initializes +a \c{git} repository and generates suitable \c{.gitignore} files (pass \c{-s\ +none} if you don't want that).| + +Let's take a look inside our new project: + +\ +$ tree hello +hello/ +├── .git/ +├── .bdep/ +├── build/ +├── hello/ +│ ├── buildfile +│ └── hello.cxx +├── buildfile +├── manifest +└── repositories.manifest +\ + +\N|While the canonical project structure is strongly recommended, especially +for new projects, \c{build2} is flexible enough to allow most commonly used +arrangements.| + +Similar to version control tools, we normally run all \c{build2} tools from +the project's source directory or one of its subdirectories, so: + +\ +$ cd hello +\ + +While the project layout is discussed in more detail in later sections, let's +examine a couple of interesting files to get a sense of what's going on. We +start with the source file which should look familiar: + +\ +$ cat hello/hello.cxx + +#include <iostream> + +int main () +{ + std::cout << \"Hello, World!\" << std::endl; +} +\ + +\N|If you prefer the \c{.?pp} extensions over \c{.?xx} for your C++ source +files, pass \c{-l\ c++,cpp} to the \c{new} command. See \l{bdep-new(1)} for +details on this and other customization options.| + +Let's take a look at the accompanying \c{buildfile}: + +\ +$ cat hello/buildfile + +libs = +#import libs += libhello%lib{hello} + +exe{hello}: {hxx ixx txx cxx}{*} $libs +\ + +As the name suggests, this file describes how to build things. While its +content might look a bit cryptic, let's try infer a couple of points without +going into too much detail (the details are discussed in the following +sections). That \c{exe{hello\}} on the left of \c{:} is a \i{target} +(executable named \c{hello}) and what we have on the right are +\i{prerequisites} (C++ sources files). This \c{buildfile} uses +\l{b#name-patterns wildcard patterns} (that \c{*}) to automatically locate all +the source files. This means we don't have to edit our \c{buildfile} every +time we add a source file to our project. There also appears to be some +infrastructure for importing (commented out) and linking libraries (that +\c{libs} variable). We will see how to use it in a moment. + +Next up is \c{manifest}: + +\ +$ cat manifest +: 1 +name: hello +version: 0.1.0-a.0.z +summary: hello executable project +license: proprietary +url: https://example.org/hello +email: you@example.org +#depends: libhello >= 1.0.0 +\ + +The \c{manifest} file is what makes a build system project a \i{package}. It +contains all the metadata that a user of a package would need to know: its +name, version, license, dependencies, etc., all in one place. + +\N|Refer to \l{bpkg#manifest-format Manifest Format} for the general format of +\c{build2} manifest files and to \l{bpkg#manifest-package Package Manifest} +for details on the package manifest values.| + +As you can see, a \c{manifest} created by \l{bdep-new(1)} contains some dummy +values which you would want to adjust before publishing your package. But +let's resist the urge to adjust that strange looking \c{0.1.0-a.0.z} until we +discuss package versioning. + +\N|Next to \c{manifest} you might have noticed the \c{repositories.manifest} +file \- we will discuss its function later, when we talk about dependencies +and where they come from.| + +Project in hand, let's build it. Unlike other programming languages, C++ +development usually involves juglling a handful of build configurations: +several compilers and/or targets (\c{build2} is big on cross-compiling), +debug/release, different sanitizers and/or static analysis tools, etc. As a +result, \c{build2} is optimized for multi-configuration usage. However, as we +will see shortly, one configuration can be designated as the default with +additional conveniences. + +The \l{bdep-init(1)} command is used to initialize a project in a build +configuration. As a shortcut, it can also create new build configuration in +the process, which is just what we need here. Let's start with GCC (remember +we are in the project's root directory): + +\ +$ bdep init -C ../hello-gcc @gcc cc config.cxx=g++ +initializing project /tmp/hello/ +created configuration @gcc /tmp/hello-gcc/ (1, default) +synchronizing: + build hello/0.1.0-a.0.19700101000000 +\ + +The \cb{--create|-C} option instructs \c{init} to create a new configuration +in the specified directory. To make refering to configurations easire, we can +give it a name, which is what we do with \c{@gcc}. The next argument (\c{cc}, +stands for \i{C-common}) is the build system module we would like to configure +with \c{config.cxx=g++} being (one of) its configuration variables. Let's +ignore that \c{synchronizing:\ ...} bit for now \- it will become clear what's +going on here in a moment. + +Now the same for Clang: + +\ +$ bdep init -C ../hello-clang @clang cc config.cxx=clang++ +initializing project /tmp/hello/ +created configuration @clang /tmp/hello-clang/ (2) +synchronizing: + build hello/0.1.0-a.0.19700101000000 +\ + +If we check the parent directory, we should now see two build configurations +next to our project: + +\ +$ ls .. +hello/ +hello-gcc/ +hello-clang/ +\ + +One of the primary goals of the \c{build2} toolchain is to provide a uniform +interface across all the platforms and compilers. While the examples in this +document assume a UNIX-like operation system, they will look pretty similar if +you are on Windows. You just have to use appropriate paths, compilers, and +options. For example, to initialize our project on Windows with Visual Studio, +start the Visual Studio development command prompt and then run: + +\N|Currently we have to run \c{build2} tools from a suitable Visual Studio +development command prompt. This requirement will likely be removed in the +future.| + +\ +> bdep init -C ..\hello-debug @debug cc ^ + config.cxx=cl ^ + \"config.cc.coptions=/MDd /Z7\" ^ + config.cc.loptions=/DEBUG + +> bdep init -C ..\hello-release @release cc ^ + config.cxx=cl ^ + config.cc.coptions=/O2 +\ + +\N|Besides the \c{coptions} (compile options) and \c{loptions} (link options) +other commonly used \c{cc} module configuration variables are \c{poptions} +(preprocess options) and \c{libs} (extra libraries to link). We can also use +their \c{config.c.*} (C compilation) and \c{config.cxx.*} (C++ compilation) +variants if we only want them applied only during the respective language +compilation. For example: + +\ +$ bdep init ... cc \ + config.cxx=clang++ \ + config.cc.coptions=-g \ + config.cxx.coptions=-stdlib=libc++ +\ + +| + +One difference you might have noticed when creating the \c{gcc} and \c{clang} +configurations above is that the first one was designated as the default. The +default configuration is used by \c{bdep} commands if no configuration is +specified explicitly (see \l{bdep-projects-configs(1)} for details). It is +also the configuration that is used if we run the build system in the project +source directory. So, normally, you would make your every day development +configuration the default. Let's try that: + +\ +$ bdep status +hello configured 0.1.0-a.0.19700101000000 + +$ b +<...> + +$ hello/hello +Hello, World! +\ + +In contrast, the Clang configuration has to be requested explicitly: + +\ +$ bdep status @clang +hello configured 0.1.0-a.0.19700101000000 + +$ b ../hello-clang/hello/ +... + +$ ../hello-clang/hello/hello/hello +Hello, World! +\ + +\N|To see the actual compilation command lines run \c{b\ -v} and for even +more details, run \c{b\ -V}. See \l{b(1)} for more information on these +and other build system options.| + +While we are here, let's also check how hard it would be to cross-compile: + +\ +$ bdep init -C ../hello-mingw cc config.cxx=x86_64-w64-mingw32-g++ +initializing project /tmp/hello/ +created configuration /tmp/hello-mingw/ (1, default) +synchronizing: + build hello/0.1.0-a.0.19700101000000 + +$ b +<...> +\ + +As you can see, cross-compiling in \c{build2} is nothing special. In our case, +on a properly setup GNU/Linux machine (that automatically uses \c{wine} as an +\c{.exe} interpreter) we can even run tests: + +\ +$ b test +<...> +\ + +Let's review what it takes to initialize a project's infrastructure and +perform the first build. For an exising project: + +\ +$ git clone .../hello.git +$ cd hello +$ bdep init -C ../hello-gcc @gcc cc config.cxx=g++ +$ b +\ + +For a new project: + +\ +$ bdep new -t exe -l c++ hello +$ cd hello +$ bdep init -C ../hello-gcc @gcc cc config.cxx=g++ +$ b +\ + +If you prefer, the \c{new} and \c{init} steps can be combined into a single +command: + +\ +$ bdep new -C hello-gcc @gcc -t exe -l c++ hello cc config.cxx=g++ +\ + +Now is also a good time to get an overview of the \c{build2} toolchain. After +all, we have already used two of its tools (\c{bdep} and \c{b}) without a +clear understanding of what they actually are. + +Unlike most other programming languages that encapsulate the build system, +package dependency manager, and project dependency manager into a single tool +(such as Rust's \c{cargo} or Go's \c{go}), \c{build2} is a hierarchy of +several tools that you will be using directly and which together with your +version control system (VCS) will constitute the core of your development +toolset. + +\N|While \c{build2} can work without a VCS, the resulting functionality will +be limited significantly.| + +At the bottom of the hierarchy is the build system, \l{b(1)}. Next comes the +package dependency manager, \l{bpkg(1)}. It is primarily used for package +\i{consumption} and depends on the build system. The top of the hierarchy is +the project dependency manager, \l{bdep(1)}. It is used for project +\i{development} and relies on \c{bpkg} to provide backing for building project +packages and their dependencies. + +\N|The main reason for this separation is modularity and the resulting +flexibility: there are situations where we only need the build system (for +example, when building a package for a system package manager where all the +dependencies should be satisfied from the system repository), or only the +build system and package manager (for example, when a build bot is building a +package for CI). + +Note also that strictly speaking \c{build2} is not C/C++-specific; its build +system is general enough to handle any DAG-based operations and its +package/project dependency managers can be used for any compiled language.| + +\N|As we will see in moment, \c{build2} also integrates with your VCS in order +to automate project versioning. Note that currently only \c{git(1)} is +supported.| + +Let's now move on to the reason why there is \i{dep} in the \c{bdep} name: +dependency management. + + +\h#tour-repositories|Package Repositories| + +Say we realized that writing \i{\"Hello, World!\"} programs is a fairly common +task and that someone must have wrote a library to help with that. So let's +see if we can find something suitable to use in our project. + +Where should we look? That's a good question. But before we can try to answer +it, we need to understand where \c{build2} can source dependencies. In +\c{build2} packages come from \i{package repositories}. Two commonly used +repository types are \i{version control} and \i{archive}-based (@@ link to +repo type help topic). + +As the name suggests, a version control-based repository uses a VCS as its +ditribution mechanism. \N{Currently only \c{git} is supported.} Such a +repository normally contains multiple versions of a single package or, +perhaps, of a few related packages. + +An archive-based repository contain multiple, potentially unrelated +packages/versions as archives along with some meta information (package list, +prerequisite/complement repositories, signtures, etc) that are all accessible +via HTTP(S). + +Version control and archive-based repositories have different +tradeoffs. Version control-based repositories are great for package +developers: With services like GitHub they are trivial to setup. In fact, your +project's (already existing) VCS repository will normally be the \c{build2} +package repostiory \- you might need to add a few files, but that's about it. + +However, version control-based repositories are not without drawbacks: It will +be hard for your users to discover your packages (try searching for \"hello +library\" on GitHub \- most of the results are not even in C++ let alone +\c{build2} packages). There is also the issue of continous availability: users +can delete their repositories, services may go out of businese, etc. Version +control-based repositories also lack repository authentication and package +signing. Finally, obtainig the available packages list for such repositories +can be a slow operation. + +A central, archive-based repository would addresses all these drawbacks: It +would be a single place to search for packages. Published packages will never +disappear and can be easily mirrored. Packages are signed and the repository +is authenticated (see \l{bpkg-repository-signing(1)} for details). And, last, +but not least, it would be fast. + +\l{https://cppget.org cppget.org} is the \c{build2} community's central +package repository (which we hope one day will become \i{the C++ package +repository}). As an added benefit, packages on \l{https://cppget.org +cppget.org} are continously \l{https://cppget.org/?builds built and tested} on +all the major platform/compiler combinations with the results available as +part of the package description. + +\N|The main drawback of archive-based repositories is the setup cost. Getting +a basic repository going is relatively easy \- all you need is an HTTP(S) +server. Adding a repository web interface like that on \l{https://cppget.org +cppget.org} will require running \l{https://cppget.org/brep \c{brep}}. And +adding CI will require running a bunch of build bots +(\l{https://cppget.org/bbot \c{bbot}}).| + +\N|CI support for version control-based repositories is a work in progress.| + +To summarize, version control-based repositories are great for package +developers while a central, archive-based repository is convenient for package +consumers. A reasonable strategy is then for package developers to publish +their releases to a central repository. Package consumers can then decide +which repository to use based on their needs. For example, one could use +\l{https://cppget.org cppget.org} as a (fast, reliable, and secure) source of +stable versions but also add, say, \c{git} repositories for select packages +(perhaps with the \c{#master} fragment filter to imporive download speed) for +testing development snapshots. In this model the two repository types +complement each other. + +\N|Support for automated publishing of tagged releases to an archive-based +repository is planned.| + +@@ Show using two repos with #master? + +\h#tour-add-remove-deps|Adding and Removing Dependencies| + +Say we found \c{libhello} that we would like to use in our \c{hello} +project. First we edit the \c{repositories.manifest} file found in the root +directory of our project and add \c{libhello} repository as a prerequisite: + +\ +role: prerequisite +location: https://example.org/libhello.git +\ + +\N|Refer to \l{bpkg#manifest-repository Repository Manifest} for details on +the repository manifest values.| + +\N|If you are trying the examples for yourself and need a real repository, +then you can use \c{https://git.build2.org/hello/libhello.git}. Or you can +create and publish your own: + +\ +$ bdep new -t lib -l c++ libhello +\ + +| + +Next we edit the \c{manifest} file (again, found in the root of our project) +and specify the version constraint for \c{libhello}: + +\ +depends: libhello >= 1.0.0 (@@ ^) +\ + +Next we edit \c{hello/buildfile} and import the \c{libhello} library into our +build: + +\ +import libs += libhello%lib{hello} +\ + +Finally, we can use the library in our source code: + +\ +#include <libhello/hello.hxx> // Or import hello; + +int main () +{ + hello::say_hello (\"World\"); +} +\ + +\N|You are probably wondering why we have to specify this often repeating +information in so many places. Let's start with the source code: we can't +specify the version constraint and location there because it will have +to be repeated in every source file that uses the dependency. + +Moving up, \c{buildfile} is also not a good place to specify this information +for the same reason (a library can be imported in multiple buildfiles) plus +the build system doesn't really know anything about version constraints or +repositories which is the purview of the dependency management tools. + +Finally, we have to separate the version constraint specification and the +location specification because the same package can be present in multiple +repositories. For example, when a package from \i{version control}-based +repostiroy is published in an \i{archive}-based repository, its +\c{repositories.manifest} file is ignored and all its dependencies should be +available from the \i{archive}-based repository itself (or its fixes set of +prerequisite repositories). In other words, \c{manifest} belongs to a +package while \c{repositories.manifest} \- to a repository. + +Also note that this is unlikely to become burdensome since adding new +dependencies is not something that happens often. Plus there is a +possibility this will be automated with a \c{bdep-add(1)} command in +the future.| + +To summarize, these are the files we had to touch to add a dependency +to your project: + +\ +repositories.manifest # add https://example.org/libhello.git +manifest # add 'depends: libhello ^ 1.0.0' (@@ ^) +buildfile # import libhello +hello.cxx # use libhello +\ + +With a new dependency added, let's check status of our project: + +\ +$ bdep status +hello configured 0.1.0-a.0.19700101000000 + available 0.1.0-a.0.19700101000000#1 +\ + +The \l{bdep-status(1)} command has detected that the dependency information +has changed and tells us that a new \i{iteration} of our project (that \c{#1}) +is now available for \i{synchronization} with its build configurations. + +To synchronize a project with one or more build configurations we use the +\l{bdep-sync(1)} command: + +\ +$ bdep sync +synchronizing: + build libhello/1.0.0 (required by hello) + upgrade hello/0.1.0-a.0.19700101000000#1 +\ + +Or we could just build the project without an explicit \c{sync} \- if +necessary, it will be automatically synchronized: + +\ +$ b +synchronizing: + build libhello/1.0.0 (required by hello) + upgrade hello/0.1.0-a.0.19700101000000#1 +<...> +c++ ../hello-gcc/libhello-1.0.0/libhello/cxx{hello} +c++ hello/cxx{hello}@../hello-gcc/hello/hello/ +ld ../hello-gcc/libhello-1.0.0/libhello/libs{hello} +ld ../hello-gcc/hello/hello/exe{hello} +\ + +The synchronization as performed by the \c{sync} command is two-way: +dependency packages are first added, removed, upgraded, or downgraded in build +configurations according the project's version constraints and user +input. Then the actual versions of the dependecies present in the build +configurations are recorded in the project's \c{lockfile}. \N{The \c{lockfile} +functionality is not yet implemented.}. For a new dependency the latest +available version that satisfies the version constraint is used. + +\N|Synchronization is also the last of the \l{bdep-init(1)} command's logic.| + +Let's now examine the status in all (\c{--all|-a}) build configurations +including immediate dependencies (\c{--immediate|-i}): + +\ +$ bdep status -ai +in configuration @gcc: +hello configured 0.1.0-a.0.19700101000000#1 + libhello configured 1.0.0 + +in configuration @clang: +hello configured 0.1.0-a.0.19700101000000 + available 0.1.0-a.0.19700101000000#1 +\ + +Since we didn't specify a configuration explicitly, only the default (\c{gcc}) +was synchronized. Normally you would try a new dependency in one +configuration, make sure everything looks good, then synchronize the rest with +\c{--all|-a}. Here are a few examples: + +\ +$ bdep sync -a +$ bdep sync @gcc @clang +$ bdep sync -c ../hello-mingw +\ + +To get rid of a dependency, we simply remove it from the two manifest files +and synchronize the project. For example, assuming \c{libhello} is no longer +in the manifests: + +\ +$ bdep status +hello configured 0.1.0-a.0.19700101000000#1 + available 0.1.0-a.0.19700101000000#2 + +$ bdep sync +synchronizing: + drop libhello/1.0.0 (unused) + upgrade hello/0.1.0-a.0.19700101000000#2 +\ + + +\h#tour-upgrade-downgrade-deps|Upgrading and Downgrading Dependencies| + +Let's say we've heard a new version of \c{libhello} was released and we +would like to try it out. To refresh the list of available dependency +versions we use the \l{bdep-fetch(1)} command (or the \c{--fetch|-f} +option to \c{status}): + +\ +$ bdep fetch +$ bdep status libhello +libhello configured 1.0.0 available [2.0.0] +\ + +To upgrade (or downgrade) dependencies we again use the \l{bdep-sync(1)} +command. We can upgrade one or more specific dependencies by listing them +as arguments to \c{sync}: + +\ +$ bdep sync libhello +synchronizing: + upgrade libhello/2.0.0 + reconfigure hello/0.1.0-a.0.19700101000000#3 +\ + +Without an explicit version or the \c{--patch|-p} option, \c{sync} will +upgrade the specified dependencies to the latest available versions. For +example, if we don't like version \c{2.0.0}, we can downgrade it back to +\c{1.0.0} by specifying the version explicitly (we pass \c{--old-available|-o} +to \c{status} to see the old versions): + +\ +$ bdep status -o libhello +libhello configured 2.0.0 available [1.0.0] (2.0.0) + +$ bdep sync libhello/1.0.0 +synchronizing: + downgrade libhello/1.0.0 + reconfigure hello/0.1.0-a.0.19700101000000#4 +\ + +Instead of specific dependecies we can also upgrade (\c{--upgrade|-u}) or +patch (\c{--patch|-p}) immediate (\c{--immediate|-i}) or all +(\c{--recursive|-r}) dependencies of our project. + +As a more realistic example, let's assume our \c{hello} project also depends +on \c{libformat} and that \c{libhello} in turn depends on \c{libprint}. Here +is the dependency tree of this project: + +\ +$ bdep status -r +hello configured 0.1.0-a.0.19700101000000#1 + libhello configured 1.0.0 + libprint configured 1.0.0 + libformat configured 1.0.0 +\ + +A typical conservative dependency management workflow for a our project would +look like this: + +\ +$ bdep status -fi # refresh and examine immediate dependencies +hello configured 0.1.0-a.0.19700101000000#1 + libhello configured 1.0.0 available 1.0.1 1.0.2 1.1.0 2.0.0 + libformat configured 1.0.0 available 1.0.1 1.0.2 1.1.0 2.0.0 + +$ bdep sync -pi # upgrade immediate to latest patch version +synchronizing: + upgrade libhello/1.0.2 + upgrade libformat/1.0.2 + reconfigure hello/0.1.0-a.0.19700101000000#2 +continue? [Y/n] y +\ + +Notice that in case of such mass upgrades you are prompted for confirmation +before anything is actually changed (unless you pass \c{--yes|-y}). + +In contrast, this would be a fairly agressive workflow where we upgrade +everything to the latest available version (version constraints permitting): + +\ +$ bdep status -fr # refresh and examine all dependencies +hello configured 0.1.0-a.0.19700101000000#1 + libhello configured 1.0.0 available 1.0.1 1.1.0 2.0.0 + libprint configured 1.0.0 available 1.0.1 1.1.0 2.0.0 + libformat configured 1.0.0 available 1.0.1 1.1.0 2.0.0 + +$ bdep sync -ur # upgrade all to latest available version +synchronizing: + upgrade libprint/2.0.0 + upgrade libhello/2.0.0 + upgrade libformat/2.0.0 + reconfigure hello/0.1.0-a.0.19700101000000#2 +continue? [Y/n] y +\ + +We can also have something in between: patch all (\c{sync\ -pr}), upgrade +immediate (\c{sync\ -ui}), or even upgrade immediate and patch the rest +(\c{sync\ -ui} followed by \c{sync\ -pr}). + +\h#tour-versioning|Versioning and Release Management| + +Let's now discuss versioning and release management (and, yes, that +strange-looking \c{0.1.0-a.0.19700101000000} we keep seeing). While a build +system project doesn't need a version and a \c{bpkg} package can use custom +versioning schemes (see \l{bpkg#package-version Package Version}), a project +managed by \c{bdep} must use \i{standard versioning}. \N{A dependency, which +is a \c{bpkg} package, need not use standard versioning.} + +Standard versioning (\i{stdver}) is a \l{https://semver.org semantic +versionsing} (\i{semver}) scheme with a more precisely defined pre-release +component and without any build metadata. + +\N|If you believe that \i{semver} is just \c{\i{major}.\i{minor}.\i{patch}}, +then in your worldview \i{stdver} would be the same as \i{semver}. In reality, +\i{semver} also allows loosly defined pre-release and build metadata +components. For example, \c{1.2.3-beta.1+build.23456} is a valid \i{semver}.| + +A standard version has the following form: + +\c{\i{major}\b{.}\i{minor}\b{.}\i{patch}[\b{-}\i{prerel}]} + +The \ci{major}, \ci{minor}, and \ci{patch} components have the same semantics +as in \i{semver}. The \ci{prerel} is used to provide \i{continuous versioning} +of our project between releases. Specifically, during development of a new +version we may want to publish several pre-releases, for example, alpha or +beta. In between those we may also want to publish a number of snapshots, for +example, for CI. With continuous versioning all these releases, pre-releases, +and snapshots are assigned unique, properly ordered versions. + +\N|Continous versioning is a fundamental building block of the \c{build2} +project depdendency management. In case of snapshots, an appropriate version +is assigned automatically in cooperation with your VCS.| + +The \ci{prerel} component for a pre-release has the following form: + +\c{(\b{a}|\b{b})\b{.}\i{num}} + +Here \cb{a} stands for alpha, \cb{b} stands for beta, and \ci{num} is the +alpha/beta number. For example: + +\ +1.1.0 # final release for 1.1.0 +1.2.0-a.1 # first alpha pre-release for 1.2.0 +1.2.0-a.1 # second alpha pre-release for 1.2.0 +1.2.0-b.2 # first beta pre-release for 1.2.0 +1.2.0 # final release for 1.2.0 +\ + +The \ci{prerel} component for a snapshot has the following form: + +\c{(\b{a}|\b{b})\b{.}\i{num}\b{.}\i{snapsn}[\b{.}\i{snapid}]} + +Where \ci{snapsn} is the snapshot sequence number and \ci{snapid} is +the snapshot id. In case of \c{git}, \ci{snapsn} is the commit timestamp +in the \c{YYYYMMDDhhmmss} form and UTC timezone while \ci{snapid} is +a 12-character abbreviated commit id. For example: + +\ +1.2.3-a.1.20180319215815.26efe301f4a7 +\ + +Notice also that a snapshot version is ordered \i{after} the corresponding +pre-release version. That is, \c{1.2.3-a.1\ <\ 1.2.3-a.1.1}. As a result, it +is customary to start the development of a new version with \c{X.Y.Z-a.0.N}, +that is, a snapshot after the (non-existent) zero'th alpha release. The +following chronologically-ordered versions illustrate a typical release flow +of a project that uses \c{git} as its VCS: + +\ +0.1.0-a.0.19700101000000 # snapshot (no commits yet) +0.1.0-a.0.20180319215815.26efe301f4a7 # snapshot (first commit) +... # more commits/snapshots +0.1.0-a.1 # pre-release (first alpha) +0.1.0-a.1.20180319221826.a6f0f41205b8 # snapshot +... # more commits/snapshots +0.1.0-a.2 # pre-release (second alpha) +0.1.0-a.2.20180319231937.b701052316c9 # snapshot +... # more commits/snapshots +0.1.0-b.1 # pre-release (first beta) +0.1.0-b.1.20180319242038.c812163417da # snapshot +... # more commits/snapshots +0.1.0 # release + +0.2.0-a.0.20180319252139.d923274528eb # snapshot (first in 0.2.0) +... +\ + +Let's see how this works in practice by publishing a couple of versions for +our \c{hello} project. For more details on standard versioning and its support +in \c{build2} refer to \l{b#module-version Version Module}. + +By now it should be clear what that \c{0.1.0-a.0.19700101000000} means \- it +is a first snapshot version of our project. Since there are no commits yet, it +has the UNIX epoch as its commit timestamp. + +As a first step, let's try to commit our project and see what changes: + +\ +$ git add . +$ git commit -m \"Start hello project\" + +$ bdep status +hello configured 0.1.0-a.0.19700101000000 + available 0.1.0-a.0.20180418054428.4cf95b919a4c +\ + +Similar to dependency information, \c{status} has detected that a new +(snapshot) version of our project is available for synchronization. + +\N|Another way to view the project's version (which works even if we are +not using \c{bdep}) is with the build system's \c{info} operation: + +\ +$ bdep info +project: hello +version: 0.1.0-a.0.20180418064031.50697bae803c +summary: hello executable project +... +\ + +| + +Let's synchronize with the default build configuration: + +\ +$ bdep sync +synchronizing: + upgrade hello/0.1.0-a.0.20180418054428.4cf95b919a4c + +$ bdep status +hello configured 0.1.0-a.0.20180418054428.4cf95b919a4c +\ + +\N|Notice that we didn't have to manually change the version anywhere. All we +had to do is commit our changes and a new snapshot version was automatically +derived by \c{build2} from the new commit. Without this automation continous +versioning would be burdensome and hardly impractical.| + +If we now make another commit, we will see a similar picture: + +\ +$ bdep status +hello configured 0.1.0-a.0.20180418054428.4cf95b919a4c + available 0.1.0-a.0.20180418064031.50697bae803c +\ + +\N|Note that you don't need to manually run \c{sync} after every commit. As +discussed earlier, you can simply run the build system to update your project +and things will get automatically synchronized if necessary.| + +Ok, time for our first release. Let's start with \c{0.1.0-a.1}. Unlike +snapshots, for pre-release as well as final releases we manually update +the version in the \c{manifest} file: + +\ +version: 0.1.0-a.1 +\ + +\N|The \c{manifest} file is the singular place where we specify the package +version with the build system's \l{b#module-version \c{version} module} +making it available in buildfiles and even source code.| + +To ensure continous versioning, this change to version must be the last commit +for this (pre-)release which itself must be immediately followed by a second +change to the version starting the development of the next (pre-)release. We +also recommend that you tag the release commit with a name in the +\c{\b{v}\i{X}.\i{Y}.\i{Z}} form. + +\N|Having regular release tag names with the \cb{v} prefix allows one to +distinguish them from other tags, for example, with wildcard patterns.| + +Here is a workflow for our example: + +\ +$ git commit -a -m \"Release version 0.1.0-a.1\" +$ git tag -a v0.1.0-a.1 -m \"Tag version 0.1.0-a.1\" +$ git push --follow-tags + +# Version 0.1.0-a.1 is now public. + +$ edit manifest # change 'version: 0.1.0-a.1.z' +$ git commit -a -m \"Change version to 0.1.0-a.1.z\" +$ git push + +# Master is now open for business. +\ + +Note that when specifying a snapshot version in \c{manifest} we use the +special \cb{z} snapshot value (for example, \c{0.1.0-a.1.z}) which is +recognized and automatically replaced by \c{build2} with, in case of \c{git}, +commit timestamp and id (refer to \l{b#module-version Version Module} for +details). + +Publishing the final release is exactly the same. For completeness, here +are the commands: + +\ +$ edit manifest # change 'version: 0.1.0' +$ git commit -a -m \"Release version 0.1.0\" +$ git tag -a v0.1.0 -m \"Tag version 0.1.0\" +$ git push --follow-tags + +$ edit manifest # change 'version: 0.2.0-a.0.z' +$ git commit -a -m \"Change version to 0.2.0-a.0.z\" +$ git push +\ + +\N|One sticky point of continous versioning is choosing the next version. +For example, above should we continue with \c{0.1.1-a.0}, \c{0.2.0-a.0}, +or \c{1.0.0-a.0}? The important rule to keep in mind is that we can jump +forward to any further version at any time and without breaking continous +versioning. But we can never jump backwards. + +For example, we can start with \c{0.2.0-a.0} but if we later realize that this +will actually be a new major release, we can easily change it to +\c{1.0.0-a.0}. As a result, the general recommendation is to start +conservatively by either incrementing the patch or the minor version +component. One reasonable strategy is to incremen the minor component and, if +required, release patch versions from a separate branch (created by branching +off from the release commit).| + +When publishing the final release you may also want to clean up now +obsolete pre-release tags. For example: + +\ +$ git tag -l 'v0.1.0-*' | xargs git push --delete origin +$ git tag -l 'v0.1.0-*' | xargs git tag --delete +\ + +\N|While at first removing such tags may seem like a bad idea, pre-releases +are by nature temporary and their use only makes sense until the final release +is published. + +Also note that having a \c{git} repository with a large number of published +but unused references may result in a significant download overhead.| + + +\h#tour-consume-pkg|Package Consumption| + +Ok, now that we have published a few releases of \c{hello}, how would the +users of our project get them? While they could clone the repository and use +\c{bdep} just like we did, this is more of a development rather than +consumption workflow. For consumption it is much easier to use the package +dependency manager, \l{bpkg(1)}, directly. + +First we create a suitable build configuration with the \l{bpkg-cfg-create(1)} +command. We can use the same place for building all our tools so let's call +the directory \c{tool-builds/}. Seeing that we are only interested in using +(rather than developing) such tools, let's build them optimized and also +configure a suitable installation location: + +\ +$ bpkg create -d tool-builds cc \ + config.cxx=g++ \ + config.cc.coptions=-O3 \ + config.install.root=/usr/local \ + config.install.sudo=sudo + +$ cd tool-builds +\ + +\N|The \c{bdep} build configurations we were creating with \c{init\ -C} are +actually \c{bpkg} build configurations. In fact, underneath, \l{bdep-init(1)} +calls \l{bpkg-cfg-create(1)}.| + +To fetch and build packages (as well as all their required dependencies) +we use the \l{bpkg-pkg-build(1)} command: + +\ +$ bpkg build https://example.org/hello.git +<...> +\ + +\N|Passing a repository URL to the \c{build} command is a shortcut to the +following sequence of commands: + +\ +$ bpkg add https://example.org/hello.git # add repository +$ bpkg fetch # fetch available packages +$ bpkg build hello # build package by name +\ + +| + +Once built, we can install the package to the location that we have specified +with the \c{config.install.root} variable using the \l{bpkg-pkg-install(1)} +command: + +\ +$ bpkg install hello +<...> +\ + +\N|If on your system the installed executables don't run because of the +unresolved shared libraries, then the easiest way to fix this is usually to use +\i{rpath}. Simply add the following configuration variable when creating the +build configuration (or as an argument to the \c{install} command): + +\ +config.bin.rpath=/usr/local/lib +\ + +| + +If we need to uninstall a previously installed package, there is the +\l{bpkg-pkg-uninstall(1)} command: + +\ +$ bpkg uninstall hello +<...> +\ + +To upgrade or downgrade packages we again use the \c{build} command. Here +is a typical upgrade workflow: + +\ +$ bpkg fetch # refresh available packages +$ bpkg status # see if new versions are available + +$ bpkg uninstall hello # uninstall old version +$ bpkg build hello # upgrade to the latest version +$ bpkg install hello # install new version +\ + +Similar to \c{bdep}, to downgrade we have to specify the desired version +explicitly. There are also the \c{--upgrade|-u} and \c{--patch|-p} as well as +\c{--immediate|-i} and \c{--recursive|-r} options that allow use to upgrade or +patch packages that we have built and/or their immediate or all dependencies +(see \l{bpkg-pkg-build(1)} for details). For example, to make sure everything +is patched, run: + +\ +$ bpkg fetch +$ bpkg build -pr +\ + +If a packages are no longer needed, we can remove it from the configuration +with \l{bpkg-pkg-drop(1)}: + +\ +$ bpkg drop hello +\ + +----------------------------------------------------------------------------- + +\ +$ git clone https://git.build2.org/hello.git +$ cd hello/ + +$ bdep init --empty + +$ bdep config create @gcc ../hello-gcc/ cc config.cxx=g++ +$ bdep config create @clang ../hello-clang/ cc config.cxx=clang++ + +$ bdep init # in default (gcc) +$ bdep init @clang # in clang only +$ bdep init @clang @gcc # in clang and gcc +$ bdep init -a # in all (clang and gcc) + +$ cd ../ +$ bpkg create -d hello-mingw/ cc config.cxx=x86_64-w64-mingw32-g++ +$ bdep init -d hello/ --add hello-mingw/ + +$ tree -F ./ +... + +$ cd hello/ +$ b +$ ./hello World + +$ b ../hello-clang/hello/ +$ ../hello-clang/hello/hello World + +$ b ../hello-mingw/hello/ +$ ../hello-mingw/hello/hello.exe World + +$ edit ... +$ bdep sync # default (gcc) +$ bdep sync @clang # clang only +$ bdep sync @clang @gcc # clang and gcc +$ bdep sync -c ../hello-mingw/ # mingw +$ bdep sync -a # all (clang, gcc, and mingw) + +$ b ../hello-*/hello/ + +$ bdep fetch # default (but can change with @/-a/-c) +$ bdep status # ditto +$ bdep upgrade libhello/1.1.0 # ditto + +$ b +$ ./hello World +$ b test + +# Looks good so upgrade the rest. +# +$ bdep upgrade -f -a libhello/1.1.0 + +$ b test: ../hello-*/hello/ +\ +" |