Tweak database table and view schema - #3805
Open
AntsyLich wants to merge 11 commits into
Open
Conversation
Rebuild `mangas` as `manga` with a UNIQUE(source_id, remote_url) constraint, `remote_`/`user_`/`state_` column prefixes and `_id` -> `id`. `remote_` is what the source reported, `user_` is what the user set, `state_` is local bookkeeping that is neither; `source_id` keeps its name because it identifies the source rather than carrying its data. Entries sharing a source and url are merged rather than dropped: the survivor absorbs the details, timestamps and children of the rows it takes over, and the duplicate chapters that repointing creates are merged too, read state and history included. Drop the sync scaffolding (`version`, `is_syncing`, `favorite_modified_at`) and collapse `favorite`, `date_added` and `favorite_modified_at` into a single nullable `user_favorite_at` - null means the entry is not in the library. The child tables keep pointing at the table this replaces until their own step rebuilds them. Foreign keys are off for the whole migration, so nothing reads those clauses in between, and copying every child table here just to rewrite one clause would double the work the migration does. For the same reason the children resolve the surviving entry while they are copied rather than being rewritten here first. Merging entries is also what creates duplicate chapters and duplicate tracks, but each of those is resolved in the step that rebuilds its own table, where the constraint that makes the collision matter is introduced. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `chapters` as `chapter` with a UNIQUE(manga_id, remote_url) constraint, `remote_`/`user_`/`state_` column prefixes and `_id` -> `id`. The entries that merged in the previous step brought their chapters with them, so the chapters move onto the surviving entry here and the duplicates that creates are merged: the copy with reading history is kept, the oldest otherwise, and the read state and history of the others are folded into it. It happens here rather than with the merge because this is where the constraint that rejects the duplicates is introduced. Drop the sync scaffolding (`version`, `is_syncing`). `chapters_manga_id_index`, `chapters_unread_by_manga_index` and `idx_chapters_url` all go away: the new unique constraint indexes `manga_id`, no query filters on unread chapters, and the only url-only lookup has no callers. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `categories` as `category` with `_id` -> `id` and `sort` -> `order`, the name it already had everywhere above the database. `order` gains a UNIQUE constraint, since two categories sharing a position was never meaningful. `mangas_categories` is rebuilt in the migration only to point its foreign key at the new table; it gets its real rebuild in its own step. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `mangas_categories` as `manga_category`. The surrogate `_id` went unused, so the pair becomes the identity with a UNIQUE(manga_id, category_id) constraint; duplicate memberships were never meaningful. `idx_mangas_categories_manga_id` goes away because the unique constraint already indexes `manga_id` as its leading column. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `history` with `_id` -> `id`, `last_read` -> `read_at` and `time_read` -> `read_duration`; the old names read as booleans rather than the timestamp and duration they hold. `history_history_chapter_id_index` goes away because `chapter_id` is already UNIQUE and therefore indexed. A row also names the entry it belongs to now, not just the chapter. Reaching the entry meant joining through the chapter every time, and the entry is the thing history is read by. It is filled from the chapter it hangs off, and keyed to both: the entry first, then the chapter. A row whose chapter is already gone cannot name an entry and is dropped, which is what the foreign key would do to it once it is enforced again. The table is the only one here that keeps its name, so the original is renamed aside and the new one built under the real name. Building it beside the original and renaming it into place makes SQLite rewrite the stored definition, which would leave this table spelled differently from the same table on a fresh install. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `excluded_scanlators` as `excluded_scanlator`, singular like the other tables. The pair becomes the identity with a UNIQUE(manga_id, scanlator) constraint; excluding the same scanlator twice for one entry was never meaningful. `excluded_scanlators_manga_id_index` goes away because the unique constraint already indexes `manga_id` as its leading column. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `manga_sync` as `manga_track`: the table holds tracker entries, and nothing about it is a sync. `_id` -> `id` and `sync_id` -> `tracker_id`, which is what it identifies. `idx_manga_sync_manga_id` goes away because UNIQUE(manga_id, tracker_id) already indexes `manga_id` as its leading column. When entries merge, two rows for the same tracker can end up pointing at the survivor. ON CONFLICT REPLACE would drop one of them silently and keep whichever the copy happened to reach last, so the row to keep is chosen first: the one furthest read, then the one already on the survivor, then the oldest. A track is taken whole rather than merged field by field, because two rows on the same tracker can point at different remote entries. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Rebuild `sources` as `source` with `_id` -> `id` and `lang` -> `language`, spelled out like every other column here. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
The view is the Kotlin-facing shape of a row, so its columns are named the way the callers read them; `last_page_read` and `datefetch` were the only two that weren't. The migration builds every view once at its end, so this updates that definition rather than dropping and recreating it again. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Bound the chapter and category aggregates to the library. Both subqueries group over every chapter and every category membership in the database, and only the outer query drops the entries that are not in the library, so an entry that was merely browsed still pays for its chapters every time the library is read. The entries the view keeps are the only ones whose aggregates it uses, so the same filter belongs inside them, where `manga_user_favorite_at_index` can serve it. The view returns the same rows either way. On a 560k chapter database holding 400 entries among 4400, reading the library goes from ~156ms to ~23ms. Assisted-by: Claude:claude-opus-5
The view is the Kotlin-facing shape of a row, so its columns are named the way the callers read them: `source` becomes `sourceId` and `state_cover_last_modified` loses the prefix it only carried because the column it reads has one. The migration builds every view once at its end, so this updates that definition rather than dropping and recreating it again. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5
Syer10
approved these changes
Aug 18, 2026
Syer10
left a comment
Syer10
left a comment
There was a problem hiding this comment.
Nice.
The simplifying of favorite by using favoriteAt is a good idea.
Fixes all my issues with how things are named in the DB.
MajorTanya
reviewed
Aug 18, 2026
| dateUpload = dateUpload, | ||
| sourceOrder = sourceOrder, | ||
| lastModifiedAt = lastModifiedAt, | ||
| version = version, |
MajorTanya
reviewed
Aug 18, 2026
| ) AS merged | ||
| WHERE history.chapter_id = merged.keep_id; | ||
|
|
||
| -- Foreign keys are off during migrations, so the history of a discarded chapter goes explicitly |
Member
There was a problem hiding this comment.
Is this a SQLDelight thing / are they on again after migrations?
Member
|
With the merge of #3818, you'll have to rename the migration file to 15.sqm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #647