Description :

    In order to understand how to merge both of the softwares TAT  and Deja-Gnu, we first need to discover what functionalities they owns, which ones do the have in common and which one are tool-specific.

Problem statement :

The merging must preserve backward compatibility with existing TAT test suites and make it possible to integrate Deja-GNU functionalities easily and to perform TAT tests inside of the framework.

At first we will have to list the current functionnalities of both tools and detect which part are in common.

After we will have to make a choice on the way to integrate the TAT tool with Deja-GNU :

• On the one hand, we can integrate Deja-GNU functionnalities inside the TAT tool by creating an interface to perform the same test calls as usual, but adding new functionalities for a more powerful testing. In this case we also have to remove the common functionnalities from TAT because it's not an option to remove a functionnality from the OSS DejaGnu.
  

• On the other hand we can add TAT inside DejaGNU which means to work together with the developpers and maintainers of the project. This way we will have to wrap the calls to TAT in calls for Deja-Gnu using the wrapping API that the developpers said that was existing and integrate TAT code inside the Deja-GNU project for common functionnalities. If TAT owns an interesting feature that is not yet in Deja-GNU we could develop a hoock in Deja-GNU to call them and in the end if the features are interesting for Deja-GNU developpers, we could add them inside the project in a second time. The main problem with this approach seems that we can add functionalities to the Deja-GNU project, but it will be impossible to remove one if there is a need, then there would be a need to fork the project.

List the functionalities of DejaGnu :

General :

• Tcl and Expect based (iTcl would be used in the next releases).
• POSIX conforming test framework (POSIX 1003.3)
• abstraction layer to write portable tests.
• batch oriented.
• interactive programs.
• use of Tcl and Expect to create wrappers for existing testsuites.
• running inside automatic configuration tools autoconf, automake or alone with the "runtest" script.
• configurable verbosity level.
• use of (base)boards to set up different environments (set_board_info).
  
• local tests support.
  
• remote tests support thanks to Expect abilities.
• rlogin, file transfer via ftp, scp ...

More exhausive description of the runtest options :

• -all : display all test outputs (by default, runtest shows only the output of tests that produce unexpected results : FAIL, XPASS, ERROR).
• --build [string] :  the canonical config name of the build machine.
  
• --host [string] : sets up the hostname of the machine for this test. Used by the following procedures : ishost, istarget, isnative, and setupxfail.
• --host_board [string] : the name of the board to use.
• --target [string] : override the default settings.
• --debug : Turns on the expect internal debugging output. Debugging output is displayed as part of the runtest output, and logged to a file called dbg.log.
• --ignore [name(s)] : The names of specific tests to ignore.
• --objdir [path] : path is the top directory containing any auxiliary compiled test code. This defaults to ".."
• --outdir [path] : write output logs in directory path. The default is "."
• --reboot [name] : reboot the target board when runtest initializes.
• --srcdir [path] : path is the top directory for test scripts to run. runtest looks in this directory for any subdirectory whose name begins with the toolname (specified with --tool). For instance, with --toolgdb}, runtest uses tests in subdirectories gdb.* (with the usual shell-like filename expansion). If you do not use --srcdir, runtest looks for test directories under the current working directory.
• --strace [number] : turn on internal tracing for expect, to n levels deep.
• --connect [program] : connect to a target testing environment as specified by type, if the target is not the computer running runtest.
  
• --baud [number] : set the default baud rate to something other than 9600.
• --target_board [name(s)] : the list of target boards to run tests on.
• --tool[name(s)] : specifies which testsuite to run, and what initialization module to use. --tool is used only for these two purposes. It is not used to name the executable program to test. Executable tool names (and paths) are recorded in site.exp and you can override them by specifying Tcl variables on the command line.
• --tool_exec [name] : the path to the tool executable to test.
• --tool_opts [options] : a list of additional options to pass to the tool.
• --verbose (-v) : turns on more output.
• --version (-V) : Prints out the version numbers of DejaGnu, expect and Tcl, and exits without running any tests.
• --D[0-1] : Start the internal Tcl debugger.
• testfile.exp[=arg(s)] : specify the names of testsuites to run. By default, runtest runs all tests for the tool.
• tclvar=value : you can define Tcl variables for use by your test scripts in the same style used with make for environment variables.

Important files and directories :

• site.exp : The site.exp file contains various variables that DejaGnu used to determine the configuration of the program being tested. If the compilation uses autotools and autoconf, this file can be generated by the make site.exp command.
  

set tool calc
set srcdir .
set objdir /home/dgt/dejagnu.test

• testsuite folder : its the directory where the tests are stored (use --srcdir to change the default).
• tool.sum : this summary shows the names of all test files run; for each test file, one line of output from each pass command (showing status PASS or XPASS) or fail command (status FAIL or XFAIL).
• tool.log : shows any output generated by tests as well as the summary output.
• dbg.log : debug log file, with the --debug option, it is possible to request a log file showing the output from Expect itself, running in debugging mode.

Outputs :

    The outputs provided by the Deja-Gnu runtest script are POSIX 1003.3 compliant :

runtest may also display the following messages:

* * *

ERROR

List the functionalities of TAT :

TODO : not listed yet.

TODO : list of the TAT functionnalities

              and a compare functionnalities in another part.

Links :

Deja-Gnu manual : http://www.gnu.org/software/dejagnu/manual/