Cluster haskell syntax errors using regex #3

C8R15T14N · 2025-04-21T14:35:21Z

New testtypes "Haskell Syntax Test" and "Haskell Runtime Test"
Haskell Syntax Test: ready to use. Clusters syntax errors of haskell submissions using a regex-based approach.
Haskell Runtime Test: not yet implemented (is currently very similar to the already implemented DockerTest)

- implementation missing, marked with TODO@CHW and will be similar as in docker test

- extends DockerTest

- view: HaskellRuntimeTestManagerView (based on DockerTestManagerOverView) - controller: HaskellRuntimeTestManager (based on DockerTestManager)

Conflicts during rebase: src/main/java/de/tuclausthal/submissioninterface/persistence/dao/TestDAOIf.java src/main/java/de/tuclausthal/submissioninterface/persistence/dao/impl/TestDAO.java src/main/java/de/tuclausthal/submissioninterface/servlets/controller/TestManager.java

…r than the first error message

.../de/tuclausthal/submissioninterface/servlets/view/fragments/ShowHaskellSyntaxTestResult.java

csware · 2025-04-21T17:10:08Z

Wouldn't an extension/property to the DockerTest be a better approach to not duplicate a lot of code?

- DockerTestManager now also handles case of HaskellRuntimeTest (since HaskellRuntimeTest is a specialization of DockerTest) - HaskellRuntimeTestManager is kept for consistency (it is the corresponding controller servlet of HaskellRuntimeTestManagerView)

C8R15T14N · 2025-04-22T18:25:53Z

Wouldn't an extension/property to the DockerTest be a better approach to not duplicate a lot of code?

Regarding the Haskell Runtime Test:

The view servlet HaskellRuntimeTestManagerView currently contains duplicated code. I'm already working on automating the test case definition process, which involves rewriting most of this servlet. These changes are not included in this pull request.
The duplicated code in the corresponding controller servlet HaskellRuntimeTestManager can indeed be avoided (new commit b9865a1).
In all other places (persistence/datamodel/HaskellRuntimeTest and testframework/tests/impl/HaskellRuntimeTest), HaskellRuntimeTest already inherits from DockerTest. I agree, that the Haskell Runtime Test should extend the existing functionality of the Docker Test.

…t was

…les for HaskellSyntaxTest

esat553 · 2025-04-24T22:51:00Z

Wouldn't an extension/property to the DockerTest be a better approach to not duplicate a lot of code?

Regarding the Haskell Syntax Test:

HaskellSyntaxTest now extends the a modularaized version of DockerTest and thereby uses already defined functions hopefully this will reduce code duplicates.

csware · 2025-04-25T08:25:09Z

I was curious whether a new attribute for the dockertest may already solve the issue. Such attribute could later also be used to selected different images.

C8R15T14N · 2025-04-25T18:38:00Z

I was curious whether a new attribute for the dockertest may already solve the issue. Such attribute could later also be used to selected different images.

We decided to implement the HaskellSyntaxTest and HaskellRuntimeTest as independent test types (inheriting from the DockerTest) instead of adding an attribute to the DockerTest for the following reasons:

Consistent user experience and implementation: In the case of Java, the syntax test, IO test, and JUnit test are implemented as separate test types. To maintain consistency in both - the user experience and the structure of GATE - we created separate syntax and runtime tests for Haskell as well, instead of adding the whole functionality inside of the DockerTest.
Avoiding to over-complicate the DockerTest: Implementing the HaskellSyntaxTest and HaskellRuntimeTest inside of the DockerTest using a new attribute would require
- integrating the entire logic of both tests inside of the DockerTest
- adding case distinctions to choose the test logic and user interface according to the testtype stored in the attribute.
To prevent the DockerTest from becoming overly complex, we decided to use Java inheritance to avoid manual case distinctions and to separate the implementation of the different testtypes into different classes, while reusing the inherited functionality of the DockerTest to minimise duplicated code.
Since both tests are being developed as part of two separate bachelor theses, they will become more complex in the next few weeks.
Versatility of the DockerTest:
The DockerTest can be used for other programming languages as well (given that all required packages are installed in the image), since it allows the user to manually specify arbitrary testing expressions. We therefore implemented our Haskell specific extensions to the DockerTest in subclasses, which keeps the DockerTest versatile and language-independent. This way, adding further DockerTest based tests that are specifically adapted to other programming languages, is as easy as creating a new subclass and extending the generic functionality of the DockerTest.
Strictly separated implementations: These are two separate bachelor theses - Esat is working on the HaskellSyntaxTest while I'm implementing the HaskellRuntimeTest. It would therefore be desirable to separate our implementations as clearly as possible.

csware · 2025-05-07T17:26:01Z

Please make sure all files have a new line at the end.

csware

Overall, this is fine for the prototype. Please use final as much as possible.

csware · 2025-05-07T17:14:45Z

...tuclausthal/submissioninterface/testanalyzer/haskell/syntax/RegexBasedHaskellClustering.java

+
+    public RegexBasedHaskellClustering(Session session) {
+        this.session = session;
+        this.clusters = new LinkedHashMap<>();


Can't this be done in a static way?

csware · 2025-05-07T17:15:36Z

...n/java/de/tuclausthal/submissioninterface/servlets/controller/HaskellRuntimeTestManager.java

+
+    @Override
+    public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
+        getServletContext().getNamedDispatcher(DockerTestManager.class.getSimpleName()).forward(request, response);


This does not seem to be good style to just forward the calls to the DockerTestManager, i.e. this servlet is not necessary.

Having different views for the HaskellTest is okay.

This does not seem to be good style to just forward the calls to the DockerTestManager, i.e. this servlet is not necessary.

The HaskellRuntimeTestManager controller servlet requires some additional functionality that is not contained in the DockerTestManager controller servlet (this is still work in progress). My idea was to handle all actions that are specific to the haskell runtime test inside of the HaskellRuntimeTestManager, and forward all other calls to the DockerTestManager.

As an example, I just implemented a new "generateNewTestSteps" action inside the HaskellRuntimeTestManager, that automatically generates a certain number of haskell runtime testcases. All other actions are forwarded to the DockerTestManager (see new commit 402f094).

Then it should be added as soon as it is required.

Then it should be added as soon as it is required.

My choice of words may have been misleading - by saying "as an example", I didn’t mean that the "generateNewTestSteps" action is optional or merely illustrative for my idea. It is definitely required, since automatically generating haskell testcases is a major part of my bachelor thesis.

What I meant is that I expect to add more actions in the future that follow the same pattern - in that sense, "generateNewTestSteps" serves "as an example" for them.

src/main/java/de/tuclausthal/submissioninterface/servlets/controller/TestManager.java

src/main/java/de/tuclausthal/submissioninterface/servlets/view/PerformTestResultView.java

csware · 2025-05-07T17:18:53Z

src/main/java/de/tuclausthal/submissioninterface/testanalyzer/CommonErrorAnalyzer.java

+		JsonObject testOutputJson = Json.createReader(new StringReader(testResult.getTestOutput())).readObject();
+		String stderr = testOutputJson.containsKey("stderr") ? testOutputJson.getString("stderr") : "";
+
+		String keyStr = "HaskellSyntax: ";


Is the "Haskell" prefix really needed?

csware · 2025-05-07T17:19:57Z

...de/tuclausthal/submissioninterface/testanalyzer/haskell/syntax/HaskellErrorClassifierIf.java

+
+import de.tuclausthal.submissioninterface.persistence.datamodel.TestResult;
+
+public interface HaskellErrorClassifierIf {


Whats is the advantage of this interface?

The idea was to have more Classes for a Static code analysis e.g. k-means or clustering by AST but by doing the clustering statically right now the interface is obsolete

csware · 2025-05-07T17:20:51Z

src/main/java/de/tuclausthal/submissioninterface/testframework/tests/impl/DockerTest.java

-	private Path tempDir;
+	protected static final Random random = new Random();
+	protected final String separator;
+	protected Path tempDir;


Protected allows more than you think, i.e. protected allows field access from within the same package.

I didn't had that on mind. Thank you for the hint. I fixed it in the upcoming version

csware · 2025-05-07T17:23:58Z

src/main/java/de/tuclausthal/submissioninterface/testframework/tests/impl/DockerTest.java

+		return testCode.toString();
+	}
+
+	protected void debugLog(List<String> params, Path studentDir){


Why is this method necessary?

This was a way to modulize the logging for individual logging messages in subclasses but it is currently not in use so I will remove it

- the generator is still under development - added a placeholder instead

csware · 2025-05-13T10:18:04Z

src/main/java/de/tuclausthal/submissioninterface/servlets/controller/TestManager.java

+			response.sendRedirect(Util.generateRedirectURL(TaskManager.class.getSimpleName()
+					+ "?action=editTask&lecture=" + task.getTaskGroup().getLecture().getId()
+					+ "&taskid=" + task.getTaskid(), response));


Why are lecture, taskid and testid necessary?

src/main/java/de/tuclausthal/submissioninterface/persistence/datamodel/HaskellSyntaxTest.java

csware · 2025-05-13T10:26:26Z

Overall the formatting is not correct, also the order of imports.

csware · 2025-05-13T10:29:26Z

...main/java/de/tuclausthal/submissioninterface/testframework/tests/impl/HaskellSyntaxTest.java

+
+    public HaskellSyntaxTest(de.tuclausthal.submissioninterface.persistence.datamodel.HaskellSyntaxTest test) {
+        super(test);
+        separator = "#<GATE@" + random.nextLong() + "#@>#";


Why necessary?

csware · 2025-05-13T10:30:33Z

...main/java/de/tuclausthal/submissioninterface/testframework/tests/impl/HaskellSyntaxTest.java

+                set -e
+                echo '%s'
+                for file in *.hs; do
+                  ghci -ignore-dot-ghci -v0 -ferror-spans -fdiagnostics-color=never -Wall -e ":load $file" -e ":quit"


Is this properly escaped?

csware · 2025-05-13T10:35:37Z

...tuclausthal/submissioninterface/testanalyzer/haskell/syntax/RegexBasedHaskellClustering.java

+                return entry.getKey();
+            }
+        }
+        return "Sonstige Fehler";


csware · 2025-05-13T10:44:33Z

There are warnings in the project.

C8R15T14N and others added 16 commits April 21, 2025 14:02

Add HaskellDynamicClusteringTestManager view and controller

7e38df1

- implementation missing, marked with TODO@CHW and will be similar as in docker test

Add new HaskellRuntimeTest in testframework and in datamodel

051b48a

- extends DockerTest

Rename HaskellDynamicClusteringTest to HaskellRuntimeTest

bab09b9

User can now create haskell runtime tests

580aafc

Implement view and controller for haskell runtime test

b2d66ff

- view: HaskellRuntimeTestManagerView (based on DockerTestManagerOverView) - controller: HaskellRuntimeTestManager (based on DockerTestManager)

Add basic groupHaskellRuntimeTestResults()

f347855

implement new Test for syntax tests and implement first regex clsutering

35d7c95

implement the clustering for the Java Syntax Test with a decent View

92641b0

improve the clustering so that it is clustered based on pririty rathe…

4a3f9b0

…r than the first error message

add copyright declaration and remove debug console logs

4376465

Place Haskell Runtime Test immediately below Haskell Syntax Test

6f8a20c

Show correct testtypes in overview "Funktionstest der Abgaben"

6194b25

Change default title of haskell syntax/runtime test

022f645

Move haskell syntax classifier into own package

762b853

Small improvements: imports, formatting

389035c

csware reviewed Apr 21, 2025

View reviewed changes

.../de/tuclausthal/submissioninterface/servlets/view/fragments/ShowHaskellSyntaxTestResult.java Outdated Show resolved Hide resolved

C8R15T14N added 2 commits April 22, 2025 18:29

Fix wrong html closing tags

bcc716a

esat553 added 6 commits April 22, 2025 23:58

replace code duplicate with already defined function util function

3f47574

remove code duplicates

a60dfe6

reverse unnecessary code changes and focus on keeping DockerTest as i…

cdf93b3

…t was

remove unnecessary code fragments and modularizations

ee20e7b

fix bug where json is saved in a wrong format

8e50e4d

fix bug with closing div tag and reduce doubble declaration of variab…

76c7d6b

…les for HaskellSyntaxTest

Remove manual testcase definition from HaskellRuntimeTest

442748f

C8R15T14N marked this pull request as ready for review April 25, 2025 18:42

csware reviewed May 7, 2025

View reviewed changes

C8R15T14N and others added 8 commits May 8, 2025 18:18

Add new line at end of each file

8716465

Add action "generateNewTestSteps" to haskell runtime test

402f094

- the generator is still under development - added a placeholder instead

Formatting

cded4cf

Make the Haskell RegEx Testing static

63825f7

implement stricter privacy settings for Methods and variables

97596a4

delete methode for logging

474b947

fix formatting

677b2d4

Formatting

5bf8441

csware reviewed May 13, 2025

View reviewed changes

src/main/java/de/tuclausthal/submissioninterface/persistence/datamodel/HaskellSyntaxTest.java Outdated Show resolved Hide resolved

csware reviewed May 13, 2025

View reviewed changes

C8R15T14N added 4 commits May 15, 2025 23:59

Reformat using Eclipse formatter settings defined in ".settings"

0ef83d9

Formatting

bc44843

Formatting

aab38fc

Formatting

cb66082


		import de.tuclausthal.submissioninterface.persistence.datamodel.TestResult;

		public interface HaskellErrorClassifierIf {

Cluster haskell syntax errors using regex #3

Are you sure you want to change the base?

Cluster haskell syntax errors using regex #3

Uh oh!

Conversation

C8R15T14N commented Apr 21, 2025

Uh oh!

Uh oh!

csware commented Apr 21, 2025

Uh oh!

C8R15T14N commented Apr 22, 2025

Uh oh!

esat553 commented Apr 24, 2025

Uh oh!

csware commented Apr 25, 2025

Uh oh!

C8R15T14N commented Apr 25, 2025

Uh oh!

csware commented May 7, 2025

Uh oh!

csware left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

csware commented May 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

csware commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

csware commented May 13, 2025 •

edited

Loading