Skip to content
Snippets Groups Projects
Commit 56460918 authored by Janis Daniel Dähne's avatar Janis Daniel Dähne
Browse files

- changed readme to include

  - EOT end of test protocol line
  - stderr & too much output behaviour
  - corrected argument sign from # to $ (# is for the compile commands)
parent 9429154f
No related branches found
No related tags found
No related merge requests found
......@@ -100,22 +100,25 @@ A json object representing the response. If an errors occurs before or after the
# Protocol Compile Test
## Protocol Output Compile Test
empty
Or
when a compile error occurred then the output of the compiler
# Protocol Output Regex Test
## Protocol Output Regex Test
1. Line: the used mode e.g. used mode: blacklist
2. Line: the used connector e.g. used connector: &&
the next lines contains the test results for every file
# Protocol Output Black Box Test
## Protocol Output Black Box Test
1. Line: contains the arguments passed to the users program, starting with a $ sign (only if arguments were present)
all other lines (meaning different by the first character):
* '<' indicating a line that was inputted to the users program
* '>' indicating a line that was read from the users program
* 'EOT' indicates that the test ended here (written after the program terminated)
this is useful when some output is received after the test has finished
* 'error:' output from the error stream
OR
when a compile error occurred then the output of the compiler
......@@ -132,6 +135,10 @@ when a compile error occurred then the output of the compiler
}
```
# Test inputs
For the test input protocol see the section "Test Runner Interface"
## Response Codes
* 0 - all ok (no test server error), this has nothing to do with the test result itself (e.g. if a test fails then this can be 0)
......@@ -144,11 +151,13 @@ when a compile error occurred then the output of the compiler
## database scheme / table /commands
The Server needs a database where the commands are stored.
The Server needs a table where the commands are stored.
A **command** (actually commands) is used to compile (also syntax check) and execute a users program.
Normally this is the shell/cmd command to execute a compiler or interpreter against a given file (or files).
The data base needs to contain a table to store this commands (for config see section config.json file).
## PLang Table
The database needs to contain a table to store this commands (for config see section config.json file, you can specify the table name there but the table needs to exists already).
The table needs to have the following scheme
internalName (primary key) | compile | exec | hardTimeoutInMs | hardMemoryLimitInKb | hardDiskSpaceLimitInKb | extensions
......@@ -199,8 +208,7 @@ internalName (primary key) | compile | exec | gardTimeoutInMs | hardMemoryLimitI
java | \\"javac\\" \\"\\#5\\" | \\"java\\" \\"-cp\\" \\"\\#1\\" \\"\\#2\\" | 1000 | 1000 | 1000 | ["java"]
python | \\"python\\" \\"-m\\" \\"py_compile\\" \\"\\#5\\" | \\"python\\" \\"\\#5\\" | 1000 | 1000 | 1000 | ["py"]
**Reminder** do not foreget to escape ever path and arguments!
**Reminder** do not forget to escape ever path and arguments!
**The compile and execute commands will be wrapped by "..." (double quotes). In order to escap paths use \"...path...\" (see the command examples)**
......@@ -221,6 +229,11 @@ id | rootDirName | testType
--- | --- | ---
1473288516 | 1473288517 | blackBoxTest
# Adding a new programming language
To add a new programming language (plang) you need to add the appropriate commands for the plang in the plang table.
The plang is then known to the test server and can be used (if you use e.g. syndrom client-server then this server needs to be configured separately to handle/know the new plang).
## ipaccess.json file
A file to control the access to the Server. This file is required!
......@@ -355,14 +368,24 @@ The compile test do not need a test content (therefore the test content will be
## Black Box Test
Test content is an array of strings, every line is trimmed to remove the trailing and leading spaces
If the first line with text starts with a '#' sign then the following text in this string are the console arguments that should be injected when the user program is called (escaping is the via "..." like in the normal shell/cmd).
If the first line with text starts with a '**$**' sign then the following text in this string are the console arguments that should be injected when the user program is called (escaping is the via "..." like in the normal shell/cmd).
* A line starting with a '<' in meant to be written to the user programs stdin (input stream) all whitespaces before the first and after the last character will be removed (trimmed).
* A line starting with a '>' is expected to be outputted from the users program (stdout, output stream) all whitespaces before the first and after the last character will be removed (trimmed).
* A line containing just a '*' is used as a wildcard that matches every output (**event if the users program does not output anything**)
* A line starting with a '**<**' in meant to be written to the user programs stdin (input stream) all whitespaces before the first and after the last character will be removed (trimmed).
* A line starting with a '**>**' is expected to be outputted from the users program (stdout, output stream) all whitespaces before the first and after the last character will be removed (trimmed).
* A line containing just a '**\***' is used as a wildcard that matches every output
* A line containing just '**EOT**' indicates the end of the test (written after the program terminated) use this to separate output that was written after the test ended... (empty test --> program should output nothing)
* A line starting with '**error:**' is output from the error stream (this can be used as debug but will also contain exceptions)
**All other lines are ignored.**
*After every Test Protocol line the stderr stream is read (1 line) and added to the conversation if there was content. This not ensures that the stderr output is in the correct order or line!*
**After the program has terminated the trailing stderror lines are added to the conversation.**
**After this the trailing normal output lines are added to the conversation.**
#### Example
```
# 2 Arg2 "Arg mit Leerzeichen"
......
C:\xampp\php3\php.exe -S 192.168.30.56:8080 -t C:\Users\theju\Documents\WebProjects\SyndromeAll\TestServer
\ No newline at end of file
C:\xampp\php3\php.exe -S 192.168.30.56:8081 -t C:\Users\theju\Documents\WebProjects\SyndromeAll\TestServer
\ No newline at end of file
C:\xampp\php3\php.exe -S localhost:8001 -t C:\Users\theju\Documents\WebProjects\SyndromeAll\TestServer
\ No newline at end of file
C:\xampp\php3\php.exe -S localhost:8081 -t C:\Users\theju\Documents\WebProjects\SyndromeAll\TestServer
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment