Skip to content

Commit eeb53d6

Browse files
committed
Fold testcase_run.sh logic into judgedaemon.
The primary motivation is to significantly reduce the per-testcase overhead of judging a submission. Historically, with fewer test cases, the separation was carrying its weight. However, with modern problems requiring many test cases or passes, the overhead from dozens of forked simple programs (`cp`, `mv`, `chmod`, `grep` etc.) within the shell script has become a major performance bottleneck. Moving them into PHP effectively replaces them with simple system calls. This change results in a substantial performance improvement: * Roughly a 50% speed-up in total judging overhead per submission * Example: A simple C++ solution to a multi-pass problem (NWERC 2025 practice) was reduced from ~70s to ~33s end-to-end.
1 parent 6409457 commit eeb53d6

File tree

12 files changed

+512
-399
lines changed

12 files changed

+512
-399
lines changed

doc/manual/config-advanced.rst

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -223,7 +223,7 @@ To allow for problems that do not fit within the standard scheme of
223223
fixed input and/or output, DOMjudge has the possibility to change the
224224
way submissions are run and checked for correctness.
225225

226-
The back end script ``testcase_run.sh`` that handles
226+
The judgedaemon that handles
227227
the running and checking of submissions, calls separate programs
228228
for running submissions and comparison of the results. These can be
229229
specialised and adapted to the requirements per problem. For this, one
@@ -257,8 +257,7 @@ output. The validator program should not make any assumptions on its
257257
working directory.
258258

259259
For more details on writing and modifying a compare (or validator)
260-
script, see the ``boolfind_cmp`` example and the comments at the
261-
top of the file ``testcase_run.sh``.
260+
script, see the ``boolfind_cmp`` example.
262261

263262
Run programs
264263
------------

etc/judgehost-static.php.in

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ define('CHROOTDIR', '@judgehost_chrootdir@');
2020
define('RUNUSER', '@RUNUSER@');
2121
define('RUNGROUP', '@RUNGROUP@');
2222

23-
// Possible exitcodes from testcase_run.sh and their meaning.
23+
// Possible exitcodes from compile scripts and their meaning.
2424
$EXITCODES = array (
2525
0 => 'correct',
2626
101 => 'compiler-error',

example_problems/hello/submissions/no_output/test-timelimit-bug.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
* This is issue #122 and fixed now, see old description below.
44
*
55
* The reason for TIMELIMIT was that program and runguard stderr are
6-
* mixed and searched by testcase_run.sh for the string 'timelimit exceeded'.
6+
* mixed and searched by judgedaemon for the string 'timelimit exceeded'.
77
* This a minor bug that doesn't provide a team any advantages. It
88
* could be fixed by having runguard write the submission stderr to a
99
* separate file.

example_problems/hello/submissions/time_limit_exceeded/stress-test-fork-setsid.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
*
44
* Without cgroups however, this will crash the judging daemon: it
55
* forks processes and places these in a new session, such that
6-
* testcase_run cannot retrace and kill these. They are left running
6+
* the judgedaemon cannot retrace and kill these. They are left running
77
* and should be killed before restarting the judging daemon. The
88
* cgroups code can detect this because the processes will belong to the
99
* same cgroup.

example_problems/hello/submissions/time_limit_exceeded/test-fork.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
* timeout.
55
*
66
* The result should be a TIMELIMIT and the running forked programs
7-
* killed by testcase_run.
7+
* killed by the judgedaemon.
88
*
99
* @EXPECTED_RESULTS@: TIMELIMIT
1010
*/

judge/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ runpipe: runpipe.cc $(LIBOBJECTS)
2424

2525
install-judgehost:
2626
$(INSTALL_PROG) -t $(DESTDIR)$(judgehost_libjudgedir) \
27-
compile.sh build_executable.sh testcase_run.sh chroot-startstop.sh \
27+
compile.sh build_executable.sh chroot-startstop.sh \
2828
check_diff.sh evict version_check.sh
2929
$(INSTALL_DATA) -t $(DESTDIR)$(judgehost_libjudgedir) \
3030
judgedaemon.main.php run-interactive.sh

0 commit comments

Comments
 (0)