fix: emit warning for legacy target names#4905
Conversation
| , relSpread :: Maybe SpreadType | ||
| , relSelect :: [RelSelectField] | ||
| , depth :: Depth -- ^ used for aliasing | ||
| , relIsLegacyTargetNameMatch :: Bool -- ^ used to ease migration into a new version with breaking change |
There was a problem hiding this comment.
Main idea is to add an attribute to ReadPlan, that is then aggregated in App.hs with legacyWarnings to generate the logs. That keeps the Plan module pure.
While working on building this warning, I noticed that we might avoid the breaking change if we look for the filter twice in the list of embeddings:
All tests pass when I try this, including the one that revealed the breaking change, but there could be outliers that may still show a breaking change so I'd have to look for them. I think it's worth the try, but before doing that @steve-chavez do you see it as a viable alternative against this whole breaking change warning stuff? (this would still be useful for future breaking changes though) |
|
@laurenceisla I'd say go ahead if you think it's doable without a breaking change (not sure if possible). But we do need to add loadtests for these alias cases to see if the double searching you propose won't cause a noticeable regression. |
|
I think we should do the breaking change at some point anyway. The previous behavior was just wrong - once you rename it via an alias, that's the name it should go by. If you want another release with a warning etc. I'd be OK with that. But I'd also be OK without that added complexity and the status quo (the breaking fix as it is currently done on main). |
Right, after pondering it for some time, I believe the breaking change is inevitable. Will continue with the TODO list for this PR. |
|
Now that #4904 is closed. Can we still emit the @wolfgangwalther WDYT? |
|
@laurenceisla How about merging the first revert in another PR? I think we agree that it's not right to make the breaking change this way. Otherwise we risk this going into a new release somehow. |
As I wrote in an earlier comment, I am not opposed to that. I don't feel like it's necessary, but if you do, I'm ok with it. I think we should not revert it - because we will need to do it anyway, for correctness. So we better deal with it in a way that makes you feel good about it. |
afd0eb5 to
78bf1e5
Compare
laurenceisla
left a comment
There was a problem hiding this comment.
This should be ready for review now.
Loadtests. Add a test in
test/load/targets.httpthat uses the alias to see the impact of this new logic. This should be done in another PR.
Do you mean to test how it's currently working on main and to check the impact of this PR?
Just a note: The tests for the loadtest are always taken from the head branch. I.e., if you add a test in this PR, you will run it on all 3 branches under test, so you don't need a separate PR to get results from the new test. |
@laurenceisla I meant adding a new request like |
78bf1e5 to
2c00085
Compare
|
I added a loadtest and here are the results: https://github.com/PostgREST/postgrest/actions/runs/26861796165. It cannot compare to 200 HTTP responses in |
Right, I think there's no other way anyway, we'll also remove this logic on the next major. Then the slight perf drop will go away. @laurenceisla Could you rebase this PR so the |
2c00085 to
cbb508a
Compare
Correct. I found it difficult to separate the "new config" commit from the "emit warning" commit without the whole "revert"->"readd" thing, so I squashed them into a single one. In fact maybe all of the commits in this PR should be squashed when merged? |
|
@laurenceisla Yes, can you squash? 🙏 We've disabled squash/merge from github UI. |
| o@LegacyTargetNameWarningObs {} -> | ||
| when (logLevel >= LogError) $ do | ||
| logWithZTime loggerState $ observationMessages o |
There was a problem hiding this comment.
Why are we logging a warning on log level error? That doesn't make sense.
There was a problem hiding this comment.
@wolfgangwalther I thought you agreed above that this was fine?
I don't see any other way to communicate to an admin that some requests will fail on a next version. My read is that these are critical. Perhaps we should remove the WARNING: prefix?
There was a problem hiding this comment.
I agreed to emitting a warning. But a warning is emitted on log-level=warn, ofc?
How can this thing be critical, if the request actually succeeds? It really is just a warning, nothing else.
There was a problem hiding this comment.
But a warning is emitted on log-level=warn, ofc?
The point is to communicate to every user about the upcoming breaking change, if we cannot make log-level=warn the default (#4904), then this is not effective.
How can this thing be critical, if the request actually succeeds?
Because it will fail on a next postgREST upgrade.
What other options do we have to communicate this?
There was a problem hiding this comment.
What other options do we have to communicate this?
The release notes are the canonical way to communicate this. You can hint in the release notes that this is deprecated now and if people are unsure whether they depend on it, they can enable log-level=warn to see the relevant warnings.
I don't think we should go at this assuming we can communicate this to users who do not look at the release notes at all. Those are very likely not going to look at their logs either, no matter on which level you log that stuff.
Or, to put differently: I regularly look at the release notes of things when I update. I only look at the logs when something is already broken.
There was a problem hiding this comment.
Yeah, I agree.. otherwise we won't be helping users that start on the new version.
So to summarize:
- The server
WARNING:should appear onlog-level=warnandurl_use_legacy_target_names=true. - The client header
Warningshould appear when the config option isurl_use_legacy_target_names=true. This disregarding thelog-levelconfig (which is only server side). - The
url_use_legacy_target_namesshould befalseby default.
There was a problem hiding this comment.
Even though the above proposal is "correct", I don't think it solves the problem.
On the default case we're essentially only leaving the Warning header as trace but how would users catch that some requests have the Warning header? For that they'd have to have sentry/otel on the frontend, so the requests are traced. But then again we don't have sentry/otel support.
There was a problem hiding this comment.
For that they'd have to have sentry/otel on the frontend, so the requests are traced. But then again we don't have sentry/otel support.
Right. They need sentry on the frontend. But that doesn't conflict with the second statement, because "we" are the backend, right?
In general, it seems to me that you are arguing from a very specific perspective - the one of an operator who controls both the backend and frontend. But this is far from the only way how PostgREST is used, I guess? People provide APIs with it and have nothing to do with frontends.
I'd also be interested in @mkleczek's opinion of this whole thing: We are trying to manage a breaking change here. Of course, the first thing Michal is going to say "don't even introduce that breaking change!", but maybe you have more thoughts beyond that? In this case, we consider the "breaking change" a bug fix, too. We changed something that worked by accident to not work anymore - how could we best handle the fallout?
(the "don't introduce the breaking change" is not out of scope either btw: If really wanted to avoid a breaking change, I think we could keep supporting the legacy target names only in the cases where they worked before, but once there was a potential name conflict not support the old, unaliased names anymore. This comes at additional complexity, though...)
There was a problem hiding this comment.
I've read this discussion and have the following thoughts:
I am not convinced log messages (either WARN or ERROR) is the best idea. The problem is that many users skip minor versions, so they won't see any log messages anyway and bump straight into the breaking change.
The most canonical way of doing breaking changes is to transition gradually with a feature flag:
- First step: keep old and new behavior with new behavior as opt-in - this will let users affected by the bug to have it fixed.
- Second step: keep old and new behavior with new behavior as opt-out - this will give "lazy" users a transition period.
- Remove old behavior
I get that it is additional complexity and requires more planning/coordination between releases so to ease this we could skip point 1 (ie. go straight to opt-out).
cbb508a to
894cccf
Compare
|
There's another way to communicate warnings to clients: #5009 (comment) @laurenceisla Maybe you can show how that would look like? I believe we need to shorten the original message. |
894cccf to
bab60c4
Compare
|
@laurenceisla I think the message should be like: The main change is that we need to compress the URL string a bit (removing the |
Ah sorry, I thought you meant a working example. Yes, it would look like that.
Maybe we could take out the URL string altogether and just mention |
Yeah I think that should be fine. One question here, for the example above users also need to update |
Right. One idea could be to just mention them all, like: "filters, orders and limits." Not sure of a word to include them all right now. |
Agree, that sounds good enough. |
e45dc84 to
ecffd28
Compare
Adds `url_use_legacy_target_names` config which is enabled by default. It logs a WARNING when target names are used in filters instead of its alias, e.g. `table?select=alias:target(*)&target.id=eq.1`. When it's disabled, the previous example will return an error. It will only succeed if the alias is used, and it won't log the warning anymore. This is DEPRECATED and the disabled behavior will be the default for future releases.
b9712e9 to
46b22ed
Compare
laurenceisla
left a comment
There was a problem hiding this comment.
Addressed what was mentioned here #4905 (comment)
However I'm still leaving it on draft. Awaiting the follow up of #4905 (comment)
| # Testing the url-use-legacy-target-names configuration | ||
| GET http://postgrest/films?select=all_roles:roles(*),awards_2026:awards(*),technical_specs:specs(*)&roles.order=character&awards.year=eq.2026&specs.runtime=not.is.null&limit=1 | ||
| Prefer: tx=commit |
There was a problem hiding this comment.
Since this is not the default behavior anymore and is deprecated, should we remove this load test?
| - The name of an embedded table can no longer be used in filters, orders or limits if it has an alias by @steve-chavez, @laurenceisla in #4075 | ||
| + e.g. `?select=alias:table(*)&table.id=eq.1` is not possible anymore, use `?select=alias:table(*)&alias.id=eq.1` instead. | ||
| + You can bypass this breaking change by setting `url-use-legacy-target-names = true` (you will see a warning in the logs when this happens) | ||
|
|
||
| ### Deprecated | ||
|
|
||
| - Deprecate bypassing breaking change using `url-use-legacy-target-names = true` by @steve-chavez, @laurenceisla in #4075 |
There was a problem hiding this comment.
I modified the CHANGELOG accordingly, considering the fix as breaking change again.
Reverts #4104 and eases transition towards the breaking change.
When the request with the deprecated syntax comes we log a warning, like so:
Note that:
taskstothe_tasks,clientstoclientes"TODO
server-legacy-features=target-names.test/load/targets.httpthat uses the alias to see the impact of this new logic. This should be done in another PR.The warning only happens forNot planned anymore, see also: fix: emit warning for legacy target names #4905 (comment)log-level=warn, this requires Switch tolog-level=warnas default #4904 to be effective.