Simplify predicates in PushDownFilter optimizer rule
#16362
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
logical-expr
xudong963
force-pushed
the
simplify_predicates
branch
from
f12788d to
fee2358
Compare
findepi
reviewed
Jun 11, 2025
| let new_predicates = simplify_predicates(predicate)?; | ||
| if old_predicate_len != new_predicates.len() { | ||
| let Some(new_predicate) = conjunction(new_predicates) else { | ||
| return plan_err!("at least one expression exists"); |
There was a problem hiding this comment.
No, as long as the original filter's predicate is not empty, simplify_predicates will keep at least one predicate.
There was a problem hiding this comment.
The caller of that function doesn't need to hard-code this assumption.
What if simplify_predicates realizes that the only predicate is trivially true and can be removed (for example x = x and we know x is not null).
There was a problem hiding this comment.
Also, currently simplify_predicates doesn't have the ability, but it should have, make sense, I'll update it
findepi
reviewed
Jun 11, 2025
Comment on lines
+782
to
+785
| let predicate = split_conjunction_owned(filter.predicate.clone()); | ||
| let old_predicate_len = predicate.len(); | ||
| let new_predicates = simplify_predicates(predicate)?; |
There was a problem hiding this comment.
new_predicates operates on col comparison_operator literal
sometimes these are tied in conjuncts: a > -5 AND a < 5 or disjuncts: a < -5 OR a > 5
would it make sense to be able to process per-column predicates in both forms?
To do that we could have a function Expr -> captured per-column predicates and then be able to combine such functions on AND and on OR
This might be related https://github.com/trinodb/trino/blob/232916b75d415a5eb643cf922492eb8513d99aae/core/trino-main/src/main/java/io/trino/sql/planner/DomainTranslator.java#L365
There was a problem hiding this comment.
a > -5 AND a < 5 will be split to [a > -5, a < 5], it seems that the two split predicates can't be simplified, that is, we shoud keep the two.
There was a problem hiding this comment.
Sure, but if I can have a < -5 OR a > 5 then maybe I can have a < -5 OR a > 5 OR a < -10 OR a > 10. This is simplifiable.
There was a problem hiding this comment.
Okay, good point, we can do this as the following PR
xudong963
force-pushed
the
simplify_predicates
branch
from
fee2358 to
883f562
Compare
xudong963
force-pushed
the
simplify_predicates
branch
from
883f562 to
92d9658
Compare
alamb
changed the title
PushDownFilter optimizer rule
|
🤖 |
|
🤖: Benchmark completed Details
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
This PR adds predicate simplification capabilities to the
PushDownFilteroptimizer rule. Currently, when filters contain redundant predicates (likex > 5 AND x > 6), these are not simplified, leading to unnecessary work during query execution.The goal is to optimize filter predicates by removing redundant conditions and keeping only the most restrictive ones, which can improve query performance.
What changes are included in this PR?
New predicate simplification module (
simplify_predicates.rs):x = 5 AND x = 6becomesfalse)Enhanced push_down_filter.rs:
simplify_predicatesbefore applying other optimizationsAre these changes tested?
20+ test cases in
simplify_predicates.sltAre there any user-facing changes?