ES|QL: Improve generative tests for FORK [130015] #131206
Conversation
svilen-mihaylov-elastic
added
>test
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
elasticsearchmachine
added
the
Team:Search Relevance
svilen-mihaylov-elastic
changed the title
svilen-mihaylov-elastic
changed the title
svilen-mihaylov-elastic
changed the title
...c/main/java/org/elasticsearch/xpack/esql/qa/rest/generative/command/pipe/WhereGenerator.java
Show resolved
Hide resolved
| final String command = current.commandString(); | ||
|
|
||
| // Try appending new command to parent of Fork. If we successfully execute (without exception) AND still retain the same | ||
| // schema (all Fork branches must have the same schema), we append the command. |
There was a problem hiding this comment.
Is this a strict constraint? Isn't it enough that there are no type conflicts between branches?
There was a problem hiding this comment.
We would generate more interesting commands if we did not have this restriction here.
Right now AFAICS when I run this, the FORK branches contain mostly WHERE/MV_EXPAND/SORT and ENRICH sometimes.
FORK branches don't need to have the same schema - we just need to be sure that if a column is present in multiple branches, it has the same data type everywhere.
There was a problem hiding this comment.
Yes correct, the schemas do not need to be exactly the same, just the overlapping columns need to have the same type.
I spend some time thinking about this and decided to go with the simple solution (for now). I think it will be challenging to allow different schemas which have compatible types AND allow for (mostly independent) fork sub-pipelines. Particularly for non-trivial subpipelines (> 5 stages), it will be non-trivial to keep adding random stages and checking if the types remain compatible. This may lead to a lot of repetitions and discarded results which will make the test run (possibly) a lot slower.
I will update the comment., and in any case, options are open later to iterate to further on this condition to balance coverage of Fork sub-pipelines and performance of the test itself.
...rc/main/java/org/elasticsearch/xpack/esql/qa/rest/generative/command/pipe/ForkGenerator.java
Outdated
Show resolved
Hide resolved
| final String command = current.commandString(); | ||
|
|
||
| // Try appending new command to parent of Fork. If we successfully execute (without exception) AND still retain the same | ||
| // schema (all Fork branches must have the same schema), we append the command. |
There was a problem hiding this comment.
We would generate more interesting commands if we did not have this restriction here.
Right now AFAICS when I run this, the FORK branches contain mostly WHERE/MV_EXPAND/SORT and ENRICH sometimes.
FORK branches don't need to have the same schema - we just need to be sure that if a column is present in multiple branches, it has the same data type everywhere.
Addresses #130015