summaryrefslogtreecommitdiff
path: root/build2
diff options
context:
space:
mode:
authorBoris Kolpackov <boris@codesynthesis.com>2016-09-16 01:32:44 +0200
committerBoris Kolpackov <boris@codesynthesis.com>2016-09-16 01:32:44 +0200
commitda9d84913ea2f6550875533fc26feb4209aa759c (patch)
tree5362dd48ad53078256915f308da038f679021121 /build2
parenta37df99fb05010110659453bc620b10d7bea4c0d (diff)
Add feature: Test description language (testscript)
Diffstat (limited to 'build2')
-rw-r--r--build2/test/testscript111
1 files changed, 111 insertions, 0 deletions
diff --git a/build2/test/testscript b/build2/test/testscript
new file mode 100644
index 0000000..c2e2365
--- /dev/null
+++ b/build2/test/testscript
@@ -0,0 +1,111 @@
+- Test description language (testscript) [feature]
+
+Requirements
+============
+
+* Multiple testscript files (to group related test)
+
+* Run multiple tests
+
+* Specify exit status (optional, expect 0 by default).
+
+* Run pre/post test actions (setups/teardowns, e.g., database schema creation).
+
+* Test input (optional)
+
+* Test stdout output (to compare to, optional)
+
+* Test stderr output (to compare to, optional)
+
+* Test description
+
+* Create (and automatically clean up) input files
+
+? Variable expansion, looked up on target in buildfile
+
+* Ability to pass arguments from build files (how for multiple scripts?)
+ Could use $0, $1, $* if have variable expansion.
+
+Questions
+=========
+
+* What is the testcript attached at the buildfile level? An executable? What
+ if the test is not for executable. I guess we always have the directory
+ target.
+
+ We could reuse the test variable to specify (list of) scripts:
+
+ exe{hello}: test = test1 test2
+
+* Do we want a notion of multi-command test (e.g., with pre/post commands)?
+ Maybe {}? Will need it not to clean up output files too early.
+
+Notes
+=====
+
+* If expected exist status is not 0, then probably makes sense to suppress
+ stderr output if there is no comparison.
+
+Syntax
+======
+
+The goal is to be as concise as possible at least for the common cases.
+
+$0 World # hello World, 0 exit expected
+$0 "John Doe" # hello "John Doe"
+$0 == 1 # hello, 1 exit expected
+$0 != 0 # hello, non-0 exit expected
+
+$0 World >>EOO
+Hello, World!
+EOO
+
+$0 World >"Hello, World!"
+
+$0 2>>EOE
+Usage: $0 <name>
+EOE
+
+$0 --trace <<EOI >>EOO 2>>EOE # The data should appear in the order specified.
+World
+EOI
+Hello, World!
+EOO
+13 characters written
+EOE
+
+~ Built in cat command with auto-cleanup? To create files for tests, etc.
+
+~ Output file redirects should be relative to out_base. Input? Could actually
+ track created out files and use that if exists, and src otherwise. What if
+ the test create a file without a redirect? Will need to "register" it
+ somehow for auto-cleanup. Maybe &<file>?
+
+~ Will there be a way to combine stderr and stdout, could be useful sometimes
+ (e.g., order of output relative to diagnostics). 2>& syntax?
+
+~ Do we support command piping?
+
+~ What if we actually do want the output to go to a file, e.g., for a second
+ command to act on it? Maybe keep shell syntax since familiar? Or reverse it
+ (> - inline, >> - multiline, >>> to file). Simplest thing is the shortest, I
+ like it.
+
+~ $0 is target path (works out in case of a directory, but will be out).
+
+~ #-comment (also for mid-line)
+
+~ quoting & escaping (need to think and specify). Would be nice to avoid shell
+ madness. Perhaps only support escaping only inside quoted strings).
+
+~ If we support variable expansion inside "here doc", what if we need to
+ specify literal '$'? Perhaps treat heredoc as a kind of string? Maybe only
+ do effective escaping? Otherwise will have to escape backslashes...
+
+~ Will probably need to support both $foo and $(foo) syntax. Will need to
+ replicate var name lexing mode form build2. Perhaps use it to avoid
+ duplication?
+
+~ line continuation with \
+
+~ == !=