Feat/sync foundation#238
Draft
MostafaKadry wants to merge 69 commits intodevelopfrom
Draft
Conversation
Defines topology (two full ERPNext sites, branch-initiates-all-HTTPS), Sync Site Config DocType (role-dependent cardinality, username+password session auth), change capture (outbox for transactions + watermark with tombstones for masters), pluggable adapter engine, per-entity conflict rules, POS-client failover to central with three-source reconciliation, reconciliation-gated IndexedDB flush, close-shift guard, observability dashboard, security model, and test strategy. Per-entity sync details (masters, stock, transactions, conflict reconciliation) are deferred to separate sub-specs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
19 tasks covering DocTypes, sync module skeleton, custom fields, role/permissions, seeds, and Test Sync Connection button. No data flows yet — Plan 2 adds masters pull, Plan 3 adds transactions push + failover. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Introduces the Sync Site Config DocType with Branch singleton enforcement, Central multi-record support, HTTPS URL validation, branch_code format validation, and a 4-test suite covering all four scenarios. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Local multi-site bench testing needs http:// central URLs between sites on the same bench. Opt-in via env var; off by default so production enforces https://. test_https_enforced skips when the bypass is active. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Setup helpers to configure Sync Site Config on the two-bench dev topology: frappe-bench (port 8000, central) and frappe-bench-16 (port 8001, branch). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…l sections Adds §15 to umbrella spec covering: - Two-bench dev setup (frappe-bench port 8000 as central, frappe-bench-16 port 8001 as branch) - Version-agnostic sync protocol design (single codebase, runtime v15/v16 detection, pos_next-owned endpoints) - Bootstrap procedure and helper functions - Running instructions for both benches Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…flict, log, dead letter, history) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…stom fields Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The export-fixtures command in Task 17 regenerated fixture files from the dev site which was missing production roles. This restores: - custom_docperm.json: all original role permissions - role.json: POSNext Cashier + Nexus POS Manager + POS Next Sync Agent - hooks.py: merged Role fixture filter into single entry Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Tests SyncSession login from branch to central and authenticated API calls across the two-bench dev environment. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Covers central-side changes_since API, branch-side MastersPuller, first adapters (Item, ItemPrice, Customer, GenericMaster), tombstone hooks, and scheduler integration. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
10 tasks covering: changes_since API, health API, adapters (Item, ItemPrice, Customer, GenericMaster), MastersPuller engine, tombstone hooks, scheduler, test runner, and e2e integration test. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Synced data was already validated on central. Applying it on branch with full Frappe validation causes failures when linked records (Company, UOM) haven't been pulled yet or differ between v15/v16. Adds _set_sync_flags() that sets ignore_validate, ignore_links, ignore_mandatory on docs during sync apply. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Frappe's conflict detection rejects synced updates when the payload's modified timestamp differs from the local record's. Adding ignore_conflict=True fixes "has been modified after you opened it". Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…etection doc.update(payload) with central's modified timestamp causes Frappe to think the record was edited by someone else. Fix: set fields individually, skipping modified/modified_by/creation/owner. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
doc.save() triggers Company.update_default_account (v15-only method) and NestedSet validation for Item Group. Using db_update() for the update path bypasses all hooks/validations — synced data was already validated on central. Insert path still uses insert() with sync flags. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Shows last pull time, outbox stats, conflict count, watermarks table, and recent sync logs directly on the Sync Site Config form. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The scheduler worker doesn't import adapter modules by default, leaving the registry empty. Add _ensure_adapters_loaded() call before running the pull cycle. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Code review findings addressed: - Extract SKIP_ON_UPSERT constant, eliminate duplicate field-skip tuples - ItemAdapter: delegate to super() instead of duplicating upsert logic - Remove no-op serialize override from ItemAdapter - Use try/except DoesNotExistError instead of exists()+get_doc() double-tap - Auto-discover adapters via pkgutil instead of manual import list - Remove per-record frappe.db.commit() — batch commit per page - Add tombstone limit in changes_since to prevent unbounded response - Use to_payload() instead of inline as_dict() in changes.py - Cache branch_code in hooks_uuid.py (was DB query on every insert) - Log tombstone errors instead of silently swallowing - Add DIRECTIONS_PULL/DIRECTIONS_PUSH constants - Fix health.py to use enabled=1 filter Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Covers outbox hooks, OutboxDrainer, central ingest API, transaction adapters (Sales Invoice, Payment Entry, POS shifts, SLE), docstatus-aware insert pattern, and scheduler integration. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
8 tasks: SubmittableAdapter, 5 transaction adapters, outbox hooks, ingest API, OutboxDrainer with backoff/dead letter, scheduler, test runner, and e2e integration test. Includes naming series validation for branch-coded Sales Invoice names. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…OS shifts, SLE) Sales Invoice includes naming series validation against origin_branch. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Adds enqueue_to_outbox on: Sales Invoice (submit/cancel/update_after_submit), Payment Entry (submit/cancel), POS Opening/Closing Shift (submit), Stock Ledger Entry (after_insert), Customer (on_update). Adds push_if_due to cron scheduler. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Covers: glossary, architecture diagrams, data flows, configuration, naming series convention, synced doctypes registry, adapter hierarchy, all 12 DocTypes, 4 API endpoints, conflict resolution strategies, custom fields, security, monitoring, troubleshooting, file structure, and development setup. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…e per-branch masters Branch transactions (Sales Invoice, Payment Entry, POS shifts) now ingest via real doc.insert() + doc.submit() instead of raw SQL — central runs the full ERPNext flow so SLE rows, GL entries, and Bin balances are generated locally. Stock Ledger Entry sync is removed; central regenerates SLEs on submit. - Branch-prefixed naming series surfaced on Sync Site Config
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.