1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
|
- Test description language (testscript) [feature]
Requirements
============
* Run multiple tests
* Multiple testscript files (to group related test)
* Specify exit status (optional, expect 0 by default).
* Run pre/post test actions (setups/teardowns, e.g., database schema creation).
* Test input (optional)
* Test stdout output (to compare to, optional)
* Test stderr output (to compare to, optional)
* Test description
* Create (and automatically clean up) input/output files
? Variable expansion, looked up on target in buildfile
* Ability to pass arguments from build files (how for multiple scripts?)
Could use $0, $1, $*.
Questions
=========
* What if the testcript attached at the buildfile level? An executable? What
if the test is not for an executable. I guess we always have the directory
target.
We could reuse the test variable to specify (list of) scripts:
exe{hello}: test = test1 test2
* Do we want a notion of multi-command test (e.g., with pre/post commands)?
Maybe {}? Will need it not to clean up output files too early. Also var
scope.
Notes
=====
* If expected exist status is not 0, then probably makes sense to suppress
stderr output if there is no comparison.
Syntax
======
The goal is to be as concise as possible at least for the common cases.
$0 World # hello World, 0 exit expected
$0 "John Doe" # hello "John Doe"
$0 == 1 # hello, 1 exit expected
$0 != 0 # hello, non-0 exit expected
$0 World >>EOO
Hello, World!
EOO
$0 World >"Hello, World!"
$0 2>>EOE
Usage: $0 <name>
EOE
$0 --trace <<EOI >>EOO 2>>EOE # The data should appear in the order specified.
World
EOI
Hello, World!
EOO
13 characters written
EOE
~ Built-in cat command with auto-cleanup? To create files for tests, etc.
~ Built-in sed(-like) utility? Note that GCC 4.8 has broken <regex> support,
only usable from 4.9. Could be a problem.
~ We should probably allow assignment of variables and since we will also
lookup those defined in the buildfile, makes sense to use the same semantics
(and perhaps even the parser, like we do for command line). Also value
types.
So in a sense for each test we will have an "inner" variable scope where
we lookup first.
~ Could support assigning output of a program to a variable:
foo = `sed ...`
The semantics should probably be "as if you had the output literally",
without any of the extra shell splitting nonsense, just the standard
buildfile logic.
Probably also support piping.
~ Output file redirects should be relative to out_base. Input? Could actually
track created out files and use that if exists, and src otherwise. What if
the test create a file without a redirect? Will need to "register" it
somehow for auto-cleanup. Maybe &<file>?
~ Will there be a way to combine stderr and stdout, could be useful sometimes
(e.g., order of output relative to diagnostics). 2>& syntax?
~ Do we support command piping?
~ What if we actually do want the output to go to a file, e.g., for a second
command to act on it? Maybe keep shell syntax since familiar? Or reverse it
(> - inline, >> - multiline, >>> to file). Simplest thing is the shortest, I
like it.
~ $0 is target path (works out in case of a directory, but will be out).
~ #-comment (also for mid-line)
~ quoting & escaping (need to think and specify). Would be nice to avoid shell
madness. Perhaps only support escaping only inside quoted strings).
~ If we support variable expansion inside "here doc", what if we need to
specify literal '$'? Perhaps treat heredoc as a kind of string? Maybe only
do effective escaping? Otherwise will have to escape backslashes...
~ Will probably need to support both $foo and $(foo) syntax. Will need to
replicate var name lexing mode form build2. Perhaps use it to avoid
duplication?
~ line continuation with \
~ == !=
|
