feat(florida): Targeted backscrapes#7559
Open
MorganBennetDev wants to merge 2 commits into
Open
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
Fixes #7553
Summary
Adds
--skip-parsedand--scrape-uuidsoptions to the Florida backscrape command to allow targeted rescraping of failed cases.The
--skip-parsedoption will modify the existing_backfillloop to first check if a parsed file exists in S3, and if so, skip fetching full and parsing full case data. This is intended to help with broad rescrapes for cases that failed to parse in a given date range.The
--scrape-uuidsoption takes the path to a CSV file of Florida court IDs and case UUIDs. It then fetches full case data for each of these cases and saves them to S3.Once the full Florida scrape is complete and we have any necessary fixes in a new Juriscraper version, we will run the command using these options to fill in any gaps caused by parsing or network failures.
Deployment
This PR should:
skip-deploy(skips everything below)skip-web-deployskip-celery-deployskip-cronjob-deployskip-daemon-deploy