Search: allow deleting search index for a project #12584
Conversation
|
@copilot write tests for this |
There was a problem hiding this comment.
Pull Request Overview
This PR adds the ability to disable search indexing for specific projects by introducing a new search_indexing_enabled boolean field. Projects marked with this field set to False will not have their content indexed for search, helping optimize resources for projects with low search usage. Users can re-enable search indexing through the project settings form, which triggers a notification cancellation and reindexing.
- Adds
search_indexing_enabledfield to the Project model (defaults to True) - Filters out projects with disabled search indexing from all search document queries and indexing tasks
- Provides UI for users to re-enable search when it has been disabled, including automatic notification cleanup
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| readthedocs/projects/models.py | Adds new search_indexing_enabled boolean field to Project model |
| readthedocs/projects/migrations/0156_project_search_indexing_enabled.py | Migration to add the new field to both Project and HistoricalProject models |
| readthedocs/search/documents.py | Filters ProjectDocument and PageDocument queries to exclude projects with search disabled |
| readthedocs/projects/tasks/search.py | Prevents SearchIndexer creation for projects with search disabled |
| readthedocs/builds/querysets.py | Filters version reindex queries to exclude projects with search disabled |
| readthedocs/projects/forms.py | Adds form field for re-enabling search, triggers reindexing on enable, cancels notification |
| readthedocs/projects/notifications.py | Adds notification message to inform users about disabled search indexing |
| readthedocs/projects/tests/test_views.py | Tests form behavior for search_indexing_enabled field visibility and functionality |
| readthedocs/projects/tests/test_indexers.py | Tests that SearchIndexer respects the search_indexing_enabled flag |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]>
| "versioning_scheme", | ||
| "default_branch", | ||
| "readthedocs_yaml_path", | ||
| "search_indexing_enabled", |
There was a problem hiding this comment.
I'd probably like to see us migrate to a full Search settings page, instead of throwing more on this form. I don't think it'll be a ton of work, and highlights search as a first-class feature.
There was a problem hiding this comment.
At the moment this is only shown to users that have search indexing disabled, so they can rollback this change in case we make a mistake.
| type=INFO, | ||
| ), | ||
| # NOTE: Disabling search for projects and notifying users is done manually for now. | ||
| Message( |
There was a problem hiding this comment.
This message isn't created in this PR -- are we planning to enable it elsewhere?
There was a problem hiding this comment.
Yeah, I'm planning on disabling search indexing on projects with no searches and that haven't been updated in the last 3 years (103003 projects), remove them from the index and also create the notification. For .com projects without an active subscription and that haven't been updated in some months.
There was a problem hiding this comment.
This will be done via a script for now. We can consider automating this next.
This allows us to mark a project so it's never indexed for search. Currently, we will mark these projects manually and create the notification so users are aware of this change and can re-enable search indexing in case we made a mistake without contacting support.
ref https://github.com/readthedocs/meta/issues/197#issuecomment-3518456511