Skip to content

Reseed demo data: delete pages whose source files were removed#1104

Merged
JeroenDeDauw merged 2 commits into
masterfrom
feature/prune-removed-demo-pages
Jul 20, 2026
Merged

Reseed demo data: delete pages whose source files were removed#1104
JeroenDeDauw merged 2 commits into
masterfrom
feature/prune-removed-demo-pages

Conversation

@JeroenDeDauw

@JeroenDeDauw JeroenDeDauw commented Jul 20, 2026

Copy link
Copy Markdown
Member

maintenance/ImportDemoData.php only created and updated pages from the DemoData/
file tree. When a source file was removed or renamed, the previously imported wiki page
lingered forever, and the only remedy was a full make reinstall-db. The import now also
deletes previously imported pages whose source files no longer exist, so a re-import fully
reseeds the demo set — renames included.

Managed set: first-revision authorship

The pages the import may delete are exactly those whose first revision was authored by
the NeoWiki system user the maintenance script imports as. That user is used only by this
script, so first-revision authorship precisely identifies importer-created pages. Pages
created by someone else and merely overwritten by the import (for example Main_Page,
created by the installer) are never deleted.

This managed set is derived from the wiki itself, which is why there is no repo-side
tombstone or manifest list: nothing to keep in sync or forget to update, and renames are
handled automatically — the old title is absent from the current run and gets deleted, the
new title is created.

Current set and deletion

The current run records every title it imports (across the main, Schema:, Layout:,
Module: and Mapping: namespaces), reusing the action's existing title derivation.
Attempted imports count as current even when the save fails, so a transient save error
never cascades into deleting the page. After the import pass, pages in the managed set but
not the current set are deleted through MediaWiki's DeletePage machinery, performed as
the same system user with a clear reason. Normal deletion hooks run, so the graph
projection (Neo4j / SPARQL) stays in sync.

There is no opt-in flag: this is dev/demo tooling with reseed semantics and the managed-set
identification is precise. One intended consequence: on a wiki without SPARQL stores
configured, DemoData/SparqlPage/ files are not imported, so SPARQL demo pages left by an
earlier SPARQL-enabled run are pruned there — which is correct, since {{#sparql_raw}}
would render as literal wikitext on such a wiki.

Structure

Following the extension's hexagonal layering, ImportPagesAction gains two collaborator
ports — ImportedPageTitlesLookup (find previously imported page titles) and PageDeleter
(delete a page) — with MediaWiki implementations in src/Persistence/MediaWiki/. The
managed-set implementation narrows candidates to the importer's actor via the revision
table, then confirms each with RevisionLookup::getFirstRevision() rather than trusting
rev_parent_id. ImportPresenter gains deletion-progress methods wired to the maintenance
output; the maintenance script wires everything.

Covered by an orchestration test (delete what is removed, keep what is current, do not
delete on a save failure, delete nothing when unchanged, report deletion failures) and
integration tests for the managed-set lookup (first-revision discrimination, deleted and
never-authored pages) and the page deleter.

AI-authored — Claude Code, Opus 4.8 (max); detailed spec from @JeroenDeDauw, executed end-to-end without human redirection; diff not yet human-reviewed; tests written and run locally (full suite green: 1861 passing), phpcs + phpstan green, live-verified in the dev stack (file removal reseeds the page and purges its graph node, siblings survive); CI green.

JeroenDeDauw and others added 2 commits July 20, 2026 20:35
`maintenance/ImportDemoData.php` only created and updated pages from the `DemoData/`
file tree. When a source file was removed or renamed, the previously imported wiki page
lingered forever, and the only remedy was a full `make reinstall-db`. The import now also
deletes previously imported pages whose source files no longer exist, so a re-import fully
reseeds the demo set — renames included.

## Managed set: first-revision authorship

The pages the import may delete are exactly those whose *first* revision was authored by
the `NeoWiki` system user the maintenance script imports as. That user is used only by this
script, so first-revision authorship precisely identifies importer-created pages. Pages
created by someone else and merely overwritten by the import (for example `Main_Page`,
created by the installer) are never deleted.

This managed set is derived from the wiki itself, which is why there is no repo-side
tombstone or manifest list: nothing to keep in sync or forget to update, and renames are
handled automatically — the old title is absent from the current run and gets deleted, the
new title is created.

## Current set and deletion

The current run records every title it imports (across the main, `Schema:`, `Layout:`,
`Module:` and `Mapping:` namespaces), reusing the action's existing title derivation.
Attempted imports count as current even when the save fails, so a transient save error
never cascades into deleting the page. After the import pass, pages in the managed set but
not the current set are deleted through MediaWiki's `DeletePage` machinery, performed as
the same system user with a clear reason. Normal deletion hooks run, so the graph
projection (Neo4j / SPARQL) stays in sync.

There is no opt-in flag: this is dev/demo tooling with reseed semantics and the managed-set
identification is precise. One intended consequence: on a wiki without SPARQL stores
configured, `DemoData/SparqlPage/` files are not imported, so SPARQL demo pages left by an
earlier SPARQL-enabled run are pruned there — which is correct, since `{{#sparql_raw}}`
would render as literal wikitext on such a wiki.

## Structure

Following the extension's hexagonal layering, `ImportPagesAction` gains two collaborator
ports — `ImportedPageTitlesLookup` (find previously imported page titles) and `PageDeleter`
(delete a page) — with MediaWiki implementations in `src/Persistence/MediaWiki/`. The
managed-set implementation narrows candidates to the importer's actor via the `revision`
table, then confirms each with `RevisionLookup::getFirstRevision()` rather than trusting
`rev_parent_id`. `ImportPresenter` gains deletion-progress methods wired to the maintenance
output; the maintenance script wires everything.

Covered by an orchestration test (delete what is removed, keep what is current, do not
delete on a save failure, delete nothing when unchanged, report deletion failures) and
integration tests for the managed-set lookup (first-revision discrimination, deleted and
never-authored pages) and the page deleter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JeroenDeDauw
JeroenDeDauw marked this pull request as ready for review July 20, 2026 19:00
@JeroenDeDauw
JeroenDeDauw merged commit 0fee6bb into master Jul 20, 2026
6 checks passed
@JeroenDeDauw
JeroenDeDauw deleted the feature/prune-removed-demo-pages branch July 20, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant