diff options
| -rw-r--r-- | doc/intro.cli | 406 |
1 files changed, 371 insertions, 35 deletions
diff --git a/doc/intro.cli b/doc/intro.cli index 417ecc0..93e4587 100644 --- a/doc/intro.cli +++ b/doc/intro.cli @@ -140,8 +140,8 @@ $ bdep new -l c++ -t exe hello created new executable project hello in /tmp/hello/ \ -The \l{bdep-new(1)} command creates a \i{canonical} \c{build2} project. In -our case it is an executable implemented in C++. +The \l{bdep-new(1)} command creates a \c{build2} project. In this case it is +an executable implemented in C++. \N|To create a library, pass \c{-t\ lib}. By default \c{new} also initializes a \c{git} repository and generates suitable \c{.gitignore} files (pass \c{-s\ @@ -173,9 +173,10 @@ hello/ and rationale behind this layout. While it is recommended, especially for new projects, \c{build2} is flexible enough to support various arrangements used in today's C and C++ projects. Furthermore, the \l{bdep-new(1)} command -provides a number of customization options and chances are you will be able to -create your preferred layout automatically. See \l{bdep-new.xhtml#src-layout -SOURCE LAYOUT} for more information and examples.| +provides a number of customization options and chances are good you will be +able to create your preferred layout automatically. See +\l{bdep-new.xhtml#src-layout SOURCE LAYOUT} for more information and +examples.| Similar to version control tools, we normally run all \c{build2} tools from the project's source directory or one of its subdirectories, so: @@ -311,8 +312,12 @@ designated as the default with additional conveniences. The \l{bdep-init(1)} command is used to initialize a project in a build configuration. As a shortcut, it can also create a new build configuration in -the process, which is just what we need here. Let's start with GCC (remember -we are in the project's root directory): +the process, which is just what we need here. + +\N|To create build configurations separately from initialization and to manage +them after that, use the \l{bdep-config(1)} subcommands.| + +Let's start with GCC (remember we are in the project's root directory): \ $ bdep init -C ../hello-gcc @gcc cc config.cxx=g++ @@ -420,7 +425,7 @@ complete list with their rough \c{make} equivalents: We can also use their \c{config.c.*} (C compilation) and \c{config.cxx.*} (C++ compilation) variants if we only want them applied during the respective -language compilation. For example: +language compilation/linking. For example: \ $ bdep init ... cc \ @@ -530,7 +535,7 @@ project. We call it \i{deep testing}.| While we are here, let's also check how hard it would be to cross-compile: \ -$ bdep init -C ../hello-mingw @mingw cc config.cxx=x86_64-w64-mingw32-g++ +$ bdep init -C @mingw cc config.cxx=x86_64-w64-mingw32-g++ initializing in project /tmp/hello/ created configuration @mingw /tmp/hello-mingw/ auto-synchronized synchronizing: @@ -595,6 +600,24 @@ synchronizing: drop hello \ +\N|By default \c{bdep} initializes a project for development by automatically +passing \c{config.<project>.develop=true} unless a custom value is specified. +For example: + +\ +$ bdep init ... @gcc cc config.cxx=g++ config.hello.develop=false +\ + +To change the development mode of an already initialized project, use +\l{bdep-sync(1)}: + +\ +$ bdep sync @gcc config.hello.develop=false +\ + +See \l{b#proj-config Project Configuration} for background on the development +mode.| + As mentioned earlier, by default \l{bdep-new(1)} initializes a \c{git} repository for us. Now that we have successfully built and tested our project, it might be a good idea to make a first commit and publish it to a remote @@ -653,17 +676,17 @@ CI request is queued: Let's see what's going on here. By default \c{ci} submits a test request to \l{https://ci.cppget.org ci.cppget.org}, a public CI service run by the \c{build2} project (see available \l{https://ci.cppget.org?build-configs Build -Configurations} and \l{https://ci.cppget.org?ci Use Policies}). It is testing -the current working tree state (branch and commit) of our package which should -be available from our remote repository (on GitHub in this example) since -that's where the CI service expects to find it. In response we get a URL where -we can see the build and test results, logs, etc. +Configurations} and \l{https://ci.cppget.org?ci Use Policies}). In our case it +will be testing the current working tree state (branch and commit) of our +package which should be available from our remote repository (on GitHub in +this example) since that's where the CI service expects to get it from. In +response we get a URL where we can see the build and test results, logs, etc. \N|This \i{push} CI model works particularly well with the \"feature branch\" development workflow. Specifically, you would develop a new feature in a separate branch, publishing and remote-testing it as necessary. When the feature is ready, you would merge any changes from \c{master}, test the result -one more time, and then merge the feature into master.| +one more time, and then merge (fast-forward) the feature into master.| Now is a good time to get an overview of the \c{build2} toolchain. After all, we have already used two of its tools (\c{bdep} and \c{b}) without a clear @@ -679,12 +702,12 @@ management toolset. \N|While \c{build2} can work without a VCS, this will result in reduced functionality.| -At the bottom of the hierarchy is the build system, \l{b(1)}. Next comes the -package dependency manager, \l{bpkg(1)}. It is primarily used for \i{package -consumption} and depends on the build system. The top of the hierarchy is the -project dependency manager, \l{bdep(1)}. It is used for \i{project -development} and relies on \c{bpkg} for building project packages and their -dependencies. +At the bottom of the hierarchy is the \c{build2} build system, which we invoke +using the \l{b(1)} driver. Next comes the package dependency manager, +\l{bpkg(1)}. It is primarily used for \i{package consumption} and depends on +the build system. The top of the hierarchy is the project dependency manager, +\l{bdep(1)}. It is used for \i{project development} and relies on \c{bpkg} for +building project packages and their dependencies. \N|The main reason for this separation is modularity and the resulting flexibility: there are situations where we only need the build system (for @@ -706,9 +729,9 @@ configuration} (those \c{hello-gcc} and \c{hello-clang} directories). A \c{bdep} build configuration is actually a \c{bpkg} build configuration which, in the build system terms, is an \i{amalgamation} \- a project that contains \i{subprojects}. In our case, the subprojects in these amalgamations will be -the projects we have initialized with \c{init} and, as we will see later, -packages that they depend on. For example, here is what our \c{hello-gcc} -contains: +the projects we have initialized with \c{init} and, as we will see in a +moment, packages that they depend on. For example, here is what our +\c{hello-gcc} contains: \ $ tree hello-gcc @@ -730,9 +753,9 @@ meta-operation (see \l{b(1)} for details). The important point here is that the \c{bdep} build configuration is not a black box that you should never look inside of. On the contrary, it is a -normal and predictable concept of the package manager and the build system and -as long as you understand what you are doing, you should feel free to interact -with it directly.| +well-defined concept of the package manager and the build system and as long +as you understand what you are doing, you should feel free to interact with it +directly.| Let's now move on to the reason why there is \i{dep} in the \c{bdep} name: dependency management. @@ -746,8 +769,8 @@ let's see if we can find something suitable to use in our project. Where should we look? That's a good question. But before we can try to answer it, we need to understand where \c{build2} can source dependencies. In -\c{build2} packages come from \i{package repositories}. Two commonly used -repository types are \i{version control} and \i{archive}-based (see +\c{build2} packages usually come from \i{package repositories}. Two commonly +used repository types are \i{version control} and \i{archive}-based (see \l{bpkg-repository-types(1)} for details). As the name suggests, a version control-based repository uses a VCS as its @@ -756,15 +779,16 @@ repository normally contains multiple versions of a single package or, perhaps, of a few related packages. An archive-based repository contains multiple, potentially unrelated -packages/versions as archives along with some meta information (package list, +packages/versions as archives along with some metadate (package list, prerequisite/complement repositories, signatures, etc) that are all accessible via HTTP(S). Version control and archive-based repositories have different trade-offs. Version control-based repositories are great for package -developers: With services like GitHub they are trivial to setup. In fact, your -project's (already existing) VCS repository will normally be the \c{build2} -package repository \- you might need to add a few files, but that's about it. +developers since with services like GitHub they are trivial to setup. In fact, +your project's (already existing) VCS repository will normally be the +\c{build2} package repository \- you might need to add a few files, but that's +about it. However, version control-based repositories are not without drawbacks: It will be hard for your users to discover your packages (try searching for \"hello @@ -1003,7 +1027,7 @@ warning: authenticity of the certificate for pkg:cppget.org/stable cannot be established certificate is for cppget.org, \"Code Synthesis\" <admin@cppget.org> certificate SHA256 fingerprint: -86:BA:D4:DE:2C:87:1A:EE:38:<...>:5A:EA:F4:F7:8C:1D:63:30:C6 +70:64:FE:E4:E0:F3:60:F1:B4:<...>:E5:C2:68:63:4C:A6:47:39:43 trust this certificate? [y/n] y hello configured 0.1.0-a.0.19700101000000 @@ -1024,7 +1048,7 @@ the repository's about page): \ role: prerequisite location: https://pkg.cppget.org/1/stable -trust: 86:BA:D4:DE:2C:87:1A:EE:38:<...>:5A:EA:F4:F7:8C:1D:63:30:C6 +trust: 70:64:FE:E4:E0:F3:60:F1:B4:<...>:E5:C2:68:63:4C:A6:47:39:43 \ To synchronize a project with one or more build configurations we use the @@ -1244,6 +1268,310 @@ immediate (\c{sync\ -ui}), or even upgrade immediate and patch the rest (\c{sync\ -ui} followed by \c{sync\ -pr}). +\h#guide-build-time-linked|Build-Time Dependencies and Linked Configurations| + +The \c{libhello} dependency we've been playing with in the previous two +sections is a \i{runtime dependency}, that is, our \c{hello} executable needs +it at run-time. This is typical of libraries and most of our dependencies will +be of this kind. However, sometimes we may only wish to use a dependency +during the build, typically a tool, such as a source code generator. This kind +of dependency is called a \i{build-time dependency}. + +Why do we need to distinguish between the two kinds of dependencies? The +primary reason is cross-compilation: if we build a tool in the same +(cross-compiling) build configuration as our project, then we will not be able +to execute it during the build (since it's built for a different target than +what we are running). But even if you are not planning to cross-compile, there +are other good reasons: if you have multiple build configurations for your +project, you may want to share a single build of your tool between them (why +waste time building the same thing multiple times). And even if you only have +a single build of your project, you may want to build the tool with different +options (for example, optimized instead of debug). + +You can probably see where this is going: in order to properly support +build-time dependencies, we need to distinguish them from runtime and we need +an ability to build them in a separate build configuration. + +Let's see how all this works using the \l{https://cppget.org/xxd \c{xxd}} tool +as an example. If you are not familiar, \c{xxd} is a hexdump utility which can +be used to embed external binary data into C/C++ code in a portable manner. +Specifically, it can read a binary file and produce a C array definition of +its contents. For example: + +\ +$ xxd -i names.txt + +unsigned char names_txt[] = { + 0x57, 0x6f, 0x72, 0x6c, 0x64, 0x0a, 0x55, 0x6e, 0x69, 0x76, 0x65, + 0x72, 0x73, 0x65, 0x0a, 0x50, 0x65, 0x6f, 0x70, 0x6c, 0x65, 0x0a, + 0x4d, 0x61, 0x72, 0x74, 0x69, 0x61, 0x6e, 0x73, 0x0a +}; +unsigned int names_txt_len = 31; +\ + +\N|While the above output is a bit old school (using \c{unsigned int} instead +of \c{size_t}) and the array/length names are derived from the input file name +(including directories), \c{xxd} can also produce just the array values +allowing us to wrap it into an array of our choice. See the +\l{https://cppget.org/xxd \c{xxd}} package description for examples of +\c{build2} recipes that do that.| + +So here is an idea: instead of failing if the user did not specify the name to +greet, let's improve our \c{hello} program to greet a random generic name from +a pre-defined list. To make this list easier to maintain, let's keep it in a +separate file called \c{names.txt} and use \c{xxd} to embed it into our +\c{hello} executable. We can use the one name per line format, for example: + +\ +$ cat names.txt +World +Universe +People +Martians +\ + +The first step in our plan is to add a build-time dependency on \c{xxd} to our +project's \c{manifest}, similar to how we did for \c{libhello}: + +\ +... +depends: libhello ^1.0.0 +depends: * xxd >= 8.2.0 +\ + +The \c{*} mark in front of the \c{xxd} name indicates that it's a build-time +dependency (think of \c{*} as the executable mark printed by \c{ls}). + +Next we import \c{xxd} in our \c{buildfile}: + +\ +... + +import libs += libhello%lib{hello} + +import! [metadata] xxd = xxd%exe{xxd} + +... +\ + +There are two main differences compared to the way we import the \c{libhello} +library: we request metadata (\c{[metadata]}) and we do immediate importation +(\c{import!}). Let's briefly discuss what this means (for details, refer to +\l{b#intro-import Target Importation} in the build system manual). Metadata +for an executable contains information that helps the build system do a better +job when an executable is used as part of the build. For example, it includes +the uniform program name to be used for low-verbosity diagnostics as well as +the version, checksum, and environment that are used to detect changes. While +immediate importation instructs the build system to skip rule-specific +importation (for example, search for libraries in compiler-specific search +paths) and import the target here and now, failing if that's not possible. It +is usually appropriate for importing executables. Note also that the metadata +can only be requested in immediate importation. + +\N|While requesting the metadata means that you will have a simpler +\c{buildfile} and a more reliable build, it also likely means that you won't +be able to use the system-installed version of the executable since it needs +to be patched to provide the metadata.| + +Now that we have the \c{xxd} tool, let's use it from an ad hoc recipe to +convert \c{names.txt} to \c{names.cxx}. Here is the complete \c{buildfile} for +our \c{hello} executable: + +\ +libs = +import libs += libhello%lib{hello} + +import! [metadata] xxd = xxd%exe{xxd} + +exe{hello}: {hxx ixx txx cxx}{** -names} cxx{names} $libs testscript + +cxx{names}: file{names.txt} $xxd +{{ + i = $path($<[0]) + env --cwd $directory($i) -- $xxd -i $leaf($i) >$path($>) +}} +\ + +The last bit that we need do is to modify \c{hello.cxx} to use the list of +fallback names (the actual implementation is left as an exercise for the +reader): + +\ +#include <iostream> + +extern unsigned char names_txt[]; +extern unsigned int names_txt_len; + +int main (int argc, char* argv[]) +{ + using namespace std; + + if (argc < 2) + { + // TODO: pick a random name from names_txt using newline as + // a name separator. + } + + ... +} +\ + +Let's recap what we've achieved so far: we've added a build-time dependency on +\c{xxd}, we've imported it in our \c{buildfile} and used it in an ad hoc +recipe to generate \c{names.cxx}, and we've modified \c{hello.cxx} to use the +generated list of names. The only step left is to actually try to build it. +But before doing that, let's also print the list of build configurations we +currently have associated with our project (see the \c{list} subcommand +in \l{bdep-config(1)}): + +\ +$ bdep config list +@gcc /tmp/hello-gcc/ 1 target default,forwarded,auto-synchronized +@clang /tmp/hello-clang/ 2 target auto-synchronized +\ + +\ +$ b +creating configuration of host type in /tmp/hello-host/ and +associating it with project(s): + /tmp/hello/ +as if by executing command(s): + bdep config create @host --type host --no-default /tmp/hello-host \ + cc config.config.load=~host +while searching for configuration for build-time dependency xxd of +package hello/0.1.0-a.0.19700101000000#4 +while synchronizing configuration /tmp/hello-gcc/ +continue? [Y/n] y + +synchronizing /tmp/hello-gcc/: + new xxd/8.2.3075 [/tmp/hello-host/] (required by hello) + upgrade hello/0.1.0-a.0.19700101000000#4 + +c ../hello-host/xxd-8.2.3075/c{xxd} +ld ../hello-host/xxd-8.2.3075/exe{xxd} +xxd ../hello-gcc/hello/hello/cxx{names} +c++ ../hello-gcc/hello/hello/cxx{names} +c++ hello/cxx{hello}@../hello-gcc/hello/hello/ +ld ../hello-gcc/hello/hello/exe{hello} +\ + +While the diagnostics is hopefully fairly self-explanatory, let's go over the +key points. The first part goes exactly as in the previous section: because +we've added a new dependency, the build configuration needs to be synchronized +with the project state. However, this is a build-time dependency and +build-time dependencies are built in configurations of type \c{host}. So +\c{bdep} first looks for such a configuration among the configurations already +associated with the project. In our case there isn't one (from the listing +above we can see that all our configurations are of type \c{target}). In this +case, \c{bdep} offers to create one automatically. We accept this offer by +answering \c{y} at the prompt and the rest should again look familiar: the new +dependency is configured and built (but now in the host configuration) and our +project is updated (which involves running the new dependency). If we now +again print the list of build configurations associated with our project, we +will see the new configuration among them: + +\ +$ bdep config list +@gcc /tmp/hello-gcc/ 1 target default,forwarded,auto-synchronized +@clang /tmp/hello-clang/ 2 target auto-synchronized +@host /tmp/hello-host/ 3 host forwarded,auto-synchronized +\ + +Let's also try to update our project in the \c{clang} configuration: + +\ +$ bdep update @clang +synchronizing: + upgrade hello/0.1.0-a.0.19700101000000#4 + +xxd ../hello-clang/hello/hello/cxx{names} +c++ ../hello-clang/hello/hello/cxx{names} +c++ hello/cxx{hello}@../hello-clang/hello/hello/ +ld ../hello-clang/hello/hello/exe{hello} +\ + +This time we are not prompted to create another configuration nor is a new +instance of \c{xxd} gets built \- as we would have expected, the existing host +configuration with the already built \c{xxd} is reused. + +From the above output we can see that \c{bdep} creates the host configuration +using the default host compiler and build options (\c{~host}) which means the +result will most likely be optimized. But if we don't like something about the +host configuration that \c{bdep} offers us to create, we can answer \c{n} at +the prompt, create one ourselves (by perhaps copying and tweaking the command +line \c{bdep} was going to use), and then restart the build. + +Besides the \c{target} and \c{host} types, the third pre-defined configuration +type is \c{build2}, which is used for build system modules. If you would like +to try a build-time dependency on a build system module, there is a dummy +\c{libbuild2-hello} module that you can use. Simply add the following line +to your \c{manifest}: + +\ +depends: * libbuild2-hello +\ + +And the following line somewhere in your \c{buildfile}: + +\ +using hello +\ + +Then build the project and see what happens. + +\N|The \c{target} type signifies a configuration for the target or end-result +of our build. If no type is specified during the configuration creation with +the \c{--type} option (or \c{--config-type} if using \c{bdep-new}), then +\c{target} is assumed. + +The \c{host} type signifies a configuration corresponding to the host machine, +that is, the machine on which the build is performed. It is expected that an +executable built in the host configuration can be executed. Oftentimes, target +and host are the same. In this case, if you would prefer not to have separate +configurations, then you can make your target configuration \i{self-hosted} by +using the \c{host} type rather than \c{target}. For example: + +\ +$ bdep init -C ../hello-gcc @gcc --type host cc config.cxx=g++ +\ + +The \c{build2} type is a special kind of host configuration that is used to +build build system modules. It cannot be self-hosted.| + +Building build-time dependencies in separate configurations is just one +application of the more general configuration linking mechanism which allows +us to build a package in one configuration while its dependencies \- in one or +more linked configurations. This, for example, can be used to create a +\"base\" configuration with common dependencies that are shared between +multiple configurations (sometimes also referred to as build configuration +overlaying). + +Let's see how this works on our \c{hello} project. Imagine \c{libhello} that +we depend on is very big and takes a while to compile. We also aren't really +interested in building it in both \c{gcc} and \c{clang} configurations (it's +our project that we are interested in building with different compilers). +Since these two compilers are ABI-compatible (at least on Linux), we could +build \c{libhello} with just one of them and reuse the result with the +other. Let's see how we can achieve this with linked configurations (refer to +\l{bdep-config(1)} for details on subcommands involved): + +\ +$ bdep config create ../hello-base @base --no-default cc config.cxx=g++ +$ bdep config create ../hello-gcc @gcc --default cc config.cxx=g++ +$ bdep config create ../hello-clang @clang cc config.cxx=clang++ + +$ bdep config link @gcc @base +$ bdep config link @clang @base + +$ bdep init @gcc { @base }+ ?libhello +$ bdep init @clang +\ + +Most of the commands are hopefully self-explanatory except for the \c{{ +@base \}+ ?libhello} part which tells \c{bdep} to build the \c{libhello} +dependency in the \c{base} configuration (we don't have to do the same for +\c{clang} since the dependency is already built). + \h#guide-versioning-releasing|Versioning and Release Management| @@ -1992,6 +2320,14 @@ $ bpkg build hello # build package by name | +\N|If building a package involves building a build-time dependency and no +configuration of type \c{host} (or \c{build2}, if the dependency is a build +system module) is linked with the target configuration, then a private +configuration of a suitable type is automatically created and linked. See +\l{#guide-build-time-linked Build-Time Dependencies and Linked Configurations} +for background on build-time dependencies and \l{bpkg-cfg-create(1)} for more +information on \c{bpkg} configuration linking.| + Once built, we can install the package to the location that we have specified with \c{config.install.root} using the \l{bpkg-pkg-install(1)} command: |