- Test description language (testscript) [feature] Requirements ============ * Multiple testscript files (to group related test) * Run multiple tests * Specify exit status (optional, expect 0 by default). * Run pre/post test actions (setups/teardowns, e.g., database schema creation). * Test input (optional) * Test stdout output (to compare to, optional) * Test stderr output (to compare to, optional) * Test description * Create (and automatically clean up) input files ? Variable expansion, looked up on target in buildfile * Ability to pass arguments from build files (how for multiple scripts?) Could use $0, $1, $* if have variable expansion. Questions ========= * What is the testcript attached at the buildfile level? An executable? What if the test is not for executable. I guess we always have the directory target. We could reuse the test variable to specify (list of) scripts: exe{hello}: test = test1 test2 * Do we want a notion of multi-command test (e.g., with pre/post commands)? Maybe {}? Will need it not to clean up output files too early. Notes ===== * If expected exist status is not 0, then probably makes sense to suppress stderr output if there is no comparison. Syntax ====== The goal is to be as concise as possible at least for the common cases. $0 World # hello World, 0 exit expected $0 "John Doe" # hello "John Doe" $0 == 1 # hello, 1 exit expected $0 != 0 # hello, non-0 exit expected $0 World >>EOO Hello, World! EOO $0 World >"Hello, World!" $0 2>>EOE Usage: $0 EOE $0 --trace <>EOO 2>>EOE # The data should appear in the order specified. World EOI Hello, World! EOO 13 characters written EOE ~ Built in cat command with auto-cleanup? To create files for tests, etc. ~ Output file redirects should be relative to out_base. Input? Could actually track created out files and use that if exists, and src otherwise. What if the test create a file without a redirect? Will need to "register" it somehow for auto-cleanup. Maybe &? ~ Will there be a way to combine stderr and stdout, could be useful sometimes (e.g., order of output relative to diagnostics). 2>& syntax? ~ Do we support command piping? ~ What if we actually do want the output to go to a file, e.g., for a second command to act on it? Maybe keep shell syntax since familiar? Or reverse it (> - inline, >> - multiline, >>> to file). Simplest thing is the shortest, I like it. ~ $0 is target path (works out in case of a directory, but will be out). ~ #-comment (also for mid-line) ~ quoting & escaping (need to think and specify). Would be nice to avoid shell madness. Perhaps only support escaping only inside quoted strings). ~ If we support variable expansion inside "here doc", what if we need to specify literal '$'? Perhaps treat heredoc as a kind of string? Maybe only do effective escaping? Otherwise will have to escape backslashes... ~ Will probably need to support both $foo and $(foo) syntax. Will need to replicate var name lexing mode form build2. Perhaps use it to avoid duplication? ~ line continuation with \ ~ == !=