diff options
author | Karen Arutyunov <karen@codesynthesis.com> | 2020-10-10 17:22:46 +0300 |
---|---|---|
committer | Karen Arutyunov <karen@codesynthesis.com> | 2020-11-06 19:32:09 +0300 |
commit | f41599c8e9435f3dfec60b872c2b4ae31177efdd (patch) | |
tree | 088f8d9bf906e4a2ed734e034699163c9ccc7306 /doc | |
parent | ac76a4fd2afff48a0d5db84592babe5cabef3a2c (diff) |
Add support for test timeouts
Diffstat (limited to 'doc')
-rwxr-xr-x | doc/cli.sh | 1 | ||||
-rw-r--r-- | doc/manual.cli | 22 | ||||
-rw-r--r-- | doc/testscript.cli | 52 |
3 files changed, 72 insertions, 3 deletions
@@ -101,6 +101,7 @@ function compile_doc () # <file> <prefix> <suffix> --link-regex '%bpkg(#.+)?%../../bpkg/doc/build2-package-manager-manual.xhtml$1%' \ --link-regex '%bdep([-.].+)%../../bdep/doc/bdep$1%' \ --link-regex '%testscript(#.+)?%build2-testscript-manual.xhtml$1%' \ +--link-regex '%build2(#.+)?%build2-build-system-manual.xhtml$1%' \ --output-prefix "$2" \ --output-suffix "$3" \ "$1" diff --git a/doc/manual.cli b/doc/manual.cli index 73f15a5..4d4e67d 100644 --- a/doc/manual.cli +++ b/doc/manual.cli @@ -5595,6 +5595,28 @@ treated specially. It enables all the tests at and under its directory. This special treatment can be inhibited by specifying the target type explicitly (for example, \c{dir{foo/\}}). +The test execution time can be limited using the \c{config.test.timeout} +variable. Its value has the \c{<operation-timeout>/<test-timeout>} form where +the timeouts are specified in seconds and either of them (but not both) can be +omitted. The left hand side sets the timeout for the whole \c{test} operation +and the right hand side \- for individual tests. The zero value clears the +previously set timeout. For example: + +\ +b test config.test.timeout=20 # Test operation. +b test config.test.timeout=20/5 # Test operation and individual tests. +b test config.test.timeout=/5 # Individual tests. +\ + +The test timeout can be specified on multiple nested root scopes. For example, +we can specify a greater timeout for the entire build configuration and lesser +ones for individual projects. The tests must complete before the nearest of +the enclosing scope timeouts. Failed that, the timed out tests are terminated +forcibly causing the entire \c{test} operation to fail. See also the +\l{testscript#builtins-timeout \c{timeout}} builtin for specifying timeouts +from within the tests and test groups. + + \h1#module-install|\c{install} Module| \N{This chapter is a work in progress and is incomplete.} diff --git a/doc/testscript.cli b/doc/testscript.cli index d838cc0..b68844b 100644 --- a/doc/testscript.cli +++ b/doc/testscript.cli @@ -1206,7 +1206,7 @@ token is an unquoted word, then the second token of the line is examined in the \c{second_token} mode (see below). If it is a variable assignment (either \c{+=}, \c{=+}, or \c{=}), then the line type is a variable line. Otherwise, it is a test command line. Note that variables with computed names can only -be set using the \l{#builtins-set \c{set} pseudo-builtin}. +be set using the \l{#builtins-set \c{set}} pseudo-builtin. The Testscript language defines the following distinct lexing modes (or contexts): @@ -2429,10 +2429,11 @@ with a newline. \h#builtins-env|\c{env}| \ -env [-u <name>]... [-] [<name>=<value>]... -- <cmd> +env [-t <sec>] [-u <name>]... [-] [<name>=<value>]... -- <cmd> \ -Run a command adding/removing the variables to/from the environment. +Run a command limiting its execution time and/or adding/removing the variables +to/from the environment. Note that \c{env} is a \i{pseudo-builtin}. In particular, its name and the \c{--} separator must be specified \i{literally} on the command line. @@ -2449,6 +2450,11 @@ env - --unset=FOO -- $* \dl| +\li|\n\c{-t|--timeout <sec>} + + Terminate the command if it fails to complete within the specified number + of seconds. See also \l{#builtins-timeout \c{timeout}} builtin.| + \li|\n\c{-u|--unset <name>} Remove the specified variable from the environment.|| @@ -2823,6 +2829,46 @@ Test the specified \i{path} according to one of the following options. Succeed Note that tests dereference symbolic links. +\h#builtins-timeout|\c{timeout}| + +\ +timeout [-s] [<group-timeout>]/[<test-timeout>] +timeout [-s] <timeout> +\ + +Specify test and/or test group timeout. + +The first form sets the test group and/or individual test timeouts and can +only be used as a setup command. Either of the timeouts (but not both) can be +omitted. + +The second form sets the test group timeout if used as a setup or teardown +command and the remaining test fragment timeout if used as a test command. + +In both forms the timeouts are specified in seconds with the zero value +clearing the previously set timeout. + +Note that \c{timeout} is a \i{pseudo-builtin}. In particular, it must be the +only command in the pipe expression, it either succeeds or terminates +abnormally, and its standard streams cannot be redirected. + +The timeouts can be set on multiple levels: via the \c{config.test.timeout} +variable on the (potentially nested) project root scopes (see +\l{build2#module-test \c{test}} module for details), with the \c{timeout} +builtin in the nested test group scopes and the test scope, and with the +\c{env} builtin for individual commands. Each command must complete before the +nearest timeout from its timeout hierarchy. Failed that, a command is +terminated forcibly causing the entire \c{test} operation to fail unless the +expired timeout was specified with the \c{--success} option, in which case the +timed out command is assumed to have succeeded. + +\dl| + +\li|\n\c{-s|--success} + + Assume a command terminated due to this timeout to have succeeded.|| + + \h#builtins-touch|\c{touch}| \ |