mjtest issueshttps://git.scc.kit.edu/IPDSnelting/mjtest/-/issues2022-01-17T16:18:59+01:00https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/11Mjtest fails with Python 3.102022-01-17T16:18:59+01:00uxwluMjtest fails with Python 3.10@gp1285 mjtest crashes when run with Python 3.10. Example output:
```
ERROR:tests:At test case 'AocDay01.0.inputc'
Traceback (most recent call last):
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/test/tests.py",...@gp1285 mjtest crashes when run with Python 3.10. Example output:
```
ERROR:tests:At test case 'AocDay01.0.inputc'
Traceback (most recent call last):
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/test/tests.py", line 195, in _run_test_case
ret = test_case.run()
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/test/exec_tests.py", line 148, in run
out, err, rtcode = self.env.run_command("./a.out", timeout=timeout, input_bytes=input_bytes)
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/environment.py", line 190, in run_command
return execute([cmd] + list(args), timeout=timeout or self.timeout, input_bytes=input_bytes)
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/util/shell.py", line 123, in execute
(out, err, returncode) = exc.run()
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/util/shell.py", line 99, in run
self._run_process()
File "/home/i_al_istannen/Programming/Uni/comprak-tools/mjtest/mjtest/util/shell.py", line 80, in _run_process
self.proc = subprocess.Popen(self.cmd,
File "/usr/lib/python3.10/subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1843, in _execute_child
raise child_exception_type(err_msg)
subprocess.SubprocessError: Exception occurred in preexec_fn.
----------------------------------------
```https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/10Integrate CReduce directly2021-11-30T14:02:20+01:00johannes.bechbergerjohannes.bechberger@kit.eduIntegrate CReduce directlyIntegrate [CReduce](https://embed.cs.utah.edu/creduce/), so that we have a flag `--reduce` (with optional time out argument) that tells mjtest to reduce the failing test cases and output them.
The reduction takes some time, but would pr...Integrate [CReduce](https://embed.cs.utah.edu/creduce/), so that we have a flag `--reduce` (with optional time out argument) that tells mjtest to reduce the failing test cases and output them.
The reduction takes some time, but would probably reduce the bug finding effort.https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/9Add infinite loop tests for compile mode2019-01-28T16:15:17+01:00ufeblAdd infinite loop tests for compile modeTesting for infinite loops using the file ending `.inf.java` or `.inf.mj` is currently only done in the `compile-firm` and not in the `compile` mode.
Adding the `JavaInfiniteLoopTest` to the `compile` TestMode would allow us to test tha...Testing for infinite loops using the file ending `.inf.java` or `.inf.mj` is currently only done in the `compile-firm` and not in the `compile` mode.
Adding the `JavaInfiniteLoopTest` to the `compile` TestMode would allow us to test that our backend does keep infinite loops.https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/8Generation of .java_output files spuriously fails2019-01-27T11:23:40+01:00rp1446Generation of .java_output files spuriously failsThere are very simple `.java` tests that nondeterministically either
- Fail to produce a non-empty `.out` reference file
```
class HitboxTest {
public static void main(String[] args) {
Position p = new Position();
...There are very simple `.java` tests that nondeterministically either
- Fail to produce a non-empty `.out` reference file
```
class HitboxTest {
public static void main(String[] args) {
Position p = new Position();
p.x = 100;
p.y = 50;
Hitbox b = new Hitbox();
b.x = 160;
b.y = 90;
b.w = 30;
b.h = 20;
while (!b.contains(p)) {
p.move(2, 1);
}
System.out.println(p.x);
System.out.println(p.y);
}
}
class Position {
public int x;
public int y;
public void move(int xstep, int ystep) {
x = x + xstep;
y = y + ystep;
}
}
class Hitbox {
public int x;
public int y;
public int w;
public int h;
public boolean contains(Position p) {
return p.x >= x && p.x <= x + w && p.y >= y && p.y <= y + h;
}
}
```
- Have strange `javac` compile errors:
```
[FAIL ] compile:transitive_object_chain.java invalid java code, but output file missing
Report for failing test case transitive_object_chain.java
Invalid java code, but output file missing
Error code: 1
Javac error message:
[0001] transitive_object_chain.java:8: error: non-static variable out cannot be referenced from a static context
[0002] System.out.println(a.ref.ref.n);
[0003] ^
[0004] transitive_object_chain.java:8: error: method println in class System cannot be applied to given types;
[0005] System.out.println(a.ref.ref.n);
[0006] ^
[0007] required: no arguments
[0008] found: int
[0009] reason: actual and formal argument lists differ in length
[0010] 2 errors
[0011]
Source file:
[0001] class transitive_object_chain {
[0002] public static void main(String[] args) {
[0003] A_2 a = new A_2();
[0004] A_2 b = new A_2();
[0005] a.ref = b;
[0006] b.ref = a;
[0007] a.n = 5;
[0008] System.out.println(a.ref.ref.n);
[0009] }
[0010] }
[0011]
[0012] class A_2 {
[0013] public A_2 ref;
[0014] public int n;
[0015] }
[0016]
```
You can (hopefully) reproduce this by doing a `git clean -fxd` in the `tests` repository, followed by re-running `mjtest`.
Johannes (@uqddy), maybe you suspect something?uqddyuqddyhttps://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/7In `big/` werden nur Tests ohne Eingabe ausgeführt2019-01-21T21:14:33+01:00uqddyIn `big/` werden nur Tests ohne Eingabe ausgeführtuqddyuqddyhttps://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/6`all` crasht bei `compile-only`2019-01-22T16:45:15+01:00uqddy`all` crasht bei `compile-only`uqddyuqddyhttps://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/5`exec` CLI argument should error or be removed2019-01-15T17:28:14+01:00qa2270`exec` CLI argument should error or be removedAs far as I understand, the `exec` mode is only there for internal reasons, but does not actually run any tests. It should therefore not be exposed to the user. If that is not possible, it should either report an error or fall back to wh...As far as I understand, the `exec` mode is only there for internal reasons, but does not actually run any tests. It should therefore not be exposed to the user. If that is not possible, it should either report an error or fall back to what the user presumably wanted, which is the `compile` mode.uqddyuqddyhttps://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/4Provide for binary input files2018-12-18T08:41:51+01:00qa2270Provide for binary input filesThe current hot-fix for files with broken UTF-8 broke binary input files (i.e. input files which are not valid UTF-8). We should go over all the I/O and replace `str` with `bytes` and vice versa where necessary.The current hot-fix for files with broken UTF-8 broke binary input files (i.e. input files which are not valid UTF-8). We should go over all the I/O and replace `str` with `bytes` and vice versa where necessary.qa2270qa2270https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/3MJTest ignores .java.out files2019-01-17T10:37:51+01:00uieknMJTest ignores .java.out filesAs the title implies, MJTest seems to ignore .java.out files. Instead it just tests whether the compiled program has the same output as the equivalent javac-compiled program.As the title implies, MJTest seems to ignore .java.out files. Instead it just tests whether the compiled program has the same output as the equivalent javac-compiled program.https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/2Compile successful semantic tests with `--compile(-firm)`2018-11-26T12:44:23+01:00rp1446Compile successful semantic tests with `--compile(-firm)`Every test that passes the `--semantic` phase can also serve as a `--compile` or `--compile-firm` test where the runtime behavior is unspecified.
Specifically, these tests may have undefined behavior, so it doesn't even make sense to ru...Every test that passes the `--semantic` phase can also serve as a `--compile` or `--compile-firm` test where the runtime behavior is unspecified.
Specifically, these tests may have undefined behavior, so it doesn't even make sense to run the resulting executable. Yet, they might provoke crashes in the middle- or back-end, so it could still be worthwhile to test them.
Not sure what the resulting mode should be called; `compile` is already taken.
This subsumes #1, which proposes an entire new test category for this kind of testing. This proposal allows reuse of all semantic tests for this test category. https://git.scc.kit.edu/IPDSnelting/mjtest/-/issues/1Support compile only tests2018-11-26T12:44:23+01:00uqddySupport compile only testsSupport tests that only need to compile, without being actually executable.Support tests that only need to compile, without being actually executable.