feat: add OpenSearch distance operations and selectors #1335

mmiguerodriguez · 2025-10-02T13:17:04Z

No description provided.

…-operations

jgaleotti

please add test cases for the classes OpenSearchHeuristicsCalculator, ParameterExtractor and OpenSearchQueryHelper

jgaleotti · 2025-10-29T18:26:47Z

...ler/src/main/java/org/evomaster/client/java/controller/opensearch/OpenSearchQueryParser.java

+import org.evomaster.client.java.controller.opensearch.selectors.MatchSelector;
 import org.evomaster.client.java.controller.opensearch.selectors.QuerySelector;

 public class OpenSearchQueryParser {


We need tests for OpenSearchQueryParser to check that the JSON is correctly parsed into its corresponding OpenSearch operation.

jgaleotti · 2025-10-29T18:33:02Z

.../src/main/java/org/evomaster/client/java/controller/opensearch/utils/ParameterExtractor.java

+/**
+ * Utility class to extract common parameters and reduce code duplication in selectors.
+ */
+public class ParameterExtractor {


add test cases for this class

jgaleotti · 2025-10-29T18:33:21Z

...c/main/java/org/evomaster/client/java/controller/opensearch/utils/OpenSearchQueryHelper.java

+    /**
+     * Extracts the case_insensitive parameter from a term query object.
+     */
+    public static Boolean extractCaseInsensitive(Object query, String structure) {


add test cases for this class

…-operations

arcuri82

sorry for late check, but was on vacation most of last week

arcuri82 · 2025-11-18T12:10:41Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+
+        if (operation instanceof MatchOperation) {
+            return calculateDistanceForMatch((MatchOperation) operation, doc);
+        }


shouldn't these be a chain of if/else, with a final having a warn log if no match was present? (eg to see if we miss any case).

arcuri82 · 2025-11-18T12:11:56Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+            // Not enough matches - distance based on how many more matches needed
+            int shortfall = minimumRequired - matchCount;
+            // Return a distance that reflects both the shortfall and the quality of non-matching terms
+            return shortfall * 10.0 + (totalDistance / expectedTerms.size());


what if expectedTerms.size() is 0?

arcuri82 · 2025-11-18T12:14:39Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+
+        try {
+            // Create pattern with case insensitive flag if needed
+            java.util.regex.Pattern pattern;


are the regex in OpenSearch having the same syntax of Java regex? clarify here in some comments, with possible some links

OpenSearch uses Lucene regular expression syntax, which is similar but NOT identical to Java regex.

https://opensearch.org/docs/latest/query-dsl/term/regexp/
https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/util/automaton/RegExp.html

@jgaleotti Do you consider adding Java Regexp for this heuristic calculation as acceptable?

arcuri82 · 2025-11-18T12:20:31Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+
+    // Helper methods for advanced operations
+
+    private int calculateEditDistance(String s1, String s2, boolean allowTranspositions) {


geneneric helper functions on strings should not be here, but rather in client-java/distance-heuristics/src/main/java/org/evomaster/client/java/distance/heuristics/DistanceHelper.java

arcuri82 · 2025-11-18T12:24:39Z

...-tests/spring/spring-rest-opensearch/src/main/java/com/opensearch/age/OpenSearchAgeRest.java

-//    }
+    @GetMapping("gte/{gte}")
+    public List<Age> findGteAge(@PathVariable("gte") Integer gte) throws IOException {
+        return ageRepo.findGteAge(gte);


all these endpoints should return 404 if no data is found

arcuri82 · 2025-11-18T12:25:36Z

...st/java/org/evomaster/e2etests/spring/rest/opensearch/students/OpenSearchStudentsEMTest.java

                Solution<RestIndividual> solution = initAndRun(args);

-//                assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/students/{q}", null);
+                assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/students/{lastName}", null);


add assertions for both 404 and 200 cases

arcuri82 · 2025-11-18T12:27:13Z

...tests/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesRest.java

+    // Term selector tests
+    @GetMapping("category/{category}")
+    public List<Product> findByCategory(@PathVariable String category) throws IOException {
+        return productRepository.findByCategory(category);


do return 404 if nothing is found in these endpoints, as it will simplify writing assertions/checks in the tests

arcuri82 · 2025-11-18T12:28:10Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+
+                Solution<RestIndividual> solution = initAndRun(args);
+
+                assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/queries/category/{category}", null);


add assertions on 404

arcuri82 · 2025-11-18T12:31:58Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+    }
+
+    @Test
+    public void testRangeQueries() throws Throwable {


why are these E2E repeated with same configurations? can have single tests with all the assertions.
otherwise, if you only want to test specific endpoints, you have to pass args parameters to specify them, eg, with --endpointPrefix

arcuri82 · 2025-11-18T12:33:02Z

...spring/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesApp.java

@@ -0,0 +1,21 @@
+package com.opensearch.queries;


folder e2e-tests does not exist any more (we did a major refactoring of folder structures recently)

Changing that.

…-operations

mmiguerodriguez · 2025-11-18T23:26:07Z

@arcuri82 is there any known issue with the evomaster-e2e-tests-spring-rest-rsa module? Most of the times I try building the project I'm having issues with that...

[INFO] evomaster-e2e-tests-spring-rest-rsa ................ FAILURE [  0.150 s]
[INFO] evomaster-e2e-tests-spring-rpc-grpc ................ SKIPPED
[INFO] evomaster-e2e-tests-spring-rpc-thrift .............. SKIPPED
[INFO] evomaster-core-integration-tests ................... SKIPPED
[INFO] evomaster-core-driver-it ........................... SKIPPED
[INFO] evomaster-core-graphql-it .......................... SKIPPED
[INFO] evomaster-core-it .................................. SKIPPED
[INFO] evomaster-test-old-libraries ....................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Is there any way to prevent those errors?

arcuri82 · 2025-11-20T12:14:44Z

@mmiguerodriguez hi. there is no known current issue about evomaster-e2e-tests-spring-rest-rsa. what is the error you are getting?

mmiguerodriguez · 2025-11-20T13:11:46Z

I'm not completely sure what was the issue, perhaps different Java version used by Maven and IntelliJ builds...
I've modified my IntelliJ to use Java 17 and maven too. Now it's fixed.

I'll be reviewing why there's some errors on the build/tests. I've already made some of the adjustements.

arcuri82 · 2025-11-20T13:13:11Z

@mmiguerodriguez ah, yes. we moved to JDK 17 recently... and might move to 21 in the near future

mmiguerodriguez · 2025-11-20T13:13:56Z

We'll need to adjust for_developers.md since it may be outdated.

arcuri82 · 2025-11-20T13:17:37Z

@mmiguerodriguez good point! i ll do now

mmiguerodriguez added 9 commits September 23, 2025 19:55

feat: add OpenSearch distance operations and selectors

14c53c2

Merge branch 'master' into feature/os-distance-operations

df6ffe6

feature: added new integration test

a356951

Merge remote-tracking branch 'origin/master' into feature/os-distance…

9da8da8

…-operations

remove invalid assertions

54f1eac

fix tests

23b7b20

more test fixes...

b529331

fix test by using default constructor

b461a0f

Merge branch 'master' into feature/os-distance-operations

bec8eb7

mmiguerodriguez marked this pull request as ready for review October 28, 2025 21:52

jgaleotti requested changes Oct 29, 2025

View reviewed changes

mmiguerodriguez marked this pull request as draft October 29, 2025 18:36

mmiguerodriguez marked this pull request as ready for review October 29, 2025 18:36

mmiguerodriguez added 3 commits November 6, 2025 15:23

tests: add PR recommendations

41fae1f

Merge remote-tracking branch 'origin/master' into feature/os-distance…

ae37f6a

…-operations

Merge remote-tracking branch 'origin/master' into feature/os-distance…

44452ff

…-operations

jgaleotti requested a review from arcuri82 November 14, 2025 13:56

arcuri82 requested changes Nov 18, 2025

View reviewed changes

mmiguerodriguez added 10 commits November 18, 2025 10:03

Merge remote-tracking branch 'origin/master' into feature/os-distance…

f47e055

…-operations

refactor: use if/else chain with warn log for unsupported ops

dfe913b

fix: handle empty expectedTerms to prevent division by zero

5b7e4cd

docs: clarify OpenSearch regex vs Java regex syntax differences

152a361

refactor: move generic string helpers to DistanceHelper

b82ab2d

fix: return 404 when no data found in age endpoints

7b612ad

fix: return 404 in students endpoints and add 404 assertions in tests

e17daa6

fix: return 404 in queries endpoints and add 404 assertions

a874b55

refactor: move queries module to new folder structure

51ab79a

style: remove redundant javadoc comments

4bf9147


		// Helper methods for advanced operations

		private int calculateEditDistance(String s1, String s2, boolean allowTranspositions) {


		Solution<RestIndividual> solution = initAndRun(args);

		assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/queries/category/{category}", null);

feat: add OpenSearch distance operations and selectors #1335

Are you sure you want to change the base?

feat: add OpenSearch distance operations and selectors #1335

Conversation

mmiguerodriguez commented Oct 2, 2025

Uh oh!

jgaleotti left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arcuri82 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguez Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguez commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

mmiguerodriguez commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

mmiguerodriguez commented Nov 20, 2025

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mmiguerodriguez Nov 18, 2025 •

edited

Loading

mmiguerodriguez commented Nov 18, 2025 •

edited

Loading

mmiguerodriguez commented Nov 20, 2025 •

edited

Loading