-
-
Notifications
You must be signed in to change notification settings - Fork 24
Open
Description
We have, in
ServiceX/servicex_app/servicex_app/resources/transformation/submit.py
Lines 258 to 268 in 0b7f40c
| dataset_manager.refresh() | |
| # If this request has a fresh DID then submit the lookup request to the DID Finder | |
| if dataset_manager.is_lookup_required: | |
| dataset_manager.submit_lookup_request( | |
| self._generate_advertised_endpoint( | |
| "servicex/internal/transformation/" | |
| ), | |
| self.celery_app, | |
| ) | |
| request_rec.status = TransformStatus.lookup |
dataset_manager.is_lookup_required and then potentially a lookup request submission which changes the value of that property to prevent further lookups. It seems that it is currently possible for multiple processes to hit this check simultaneously for a single dataset and trigger multiple lookups (which corrupts the file records). This block needs to do a row lock on the relevant dataset.Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels