feat(overrides): GIR override system — replace hardcoded exclusion lists with sexp files#89
Merged
chris-armstrong merged 25 commits intomainfrom Apr 5, 2026
Merged
feat(overrides): GIR override system — replace hardcoded exclusion lists with sexp files#89chris-armstrong merged 25 commits intomainfrom
chris-armstrong merged 25 commits intomainfrom
Conversation
…configuration file
- override_types.ml/mli: variant types (override_action, component_override, class_override, interface_override, record_override, enum_override, bitfield_override, library_overrides) with deriving eq - override_parser.ml/mli: s-expression parser with duplicate detection, version validation, context-disambiguated (function ...) handling - types.ml: member_version, flag_version, field_version fields on gir_enum_member, gir_bitfield_member, gir_record_field - gir_parser.ml: initialize new version fields to None - test_override_types.ml: 12 tests for type construction and equality - test_override_parser.ml: 23 tests covering all entity types, error cases, duplicate detection, ignore/version overrides - Fixed ppxlib 0.35.0 API changes (Ptyp_alias, Pexp_function)
- override_apply module: applies parsed overrides to GIR data structures (entity/component ignore, version overrides, unknown-name warnings) - 24 unit tests covering all entity types, edge cases, warnings - gir_gen generate: -o flag to load override files before generation - gir_gen references: -o flag to filter ignored entities from references - Overrides applied after parsing, before type-mapping context is built
- override_apply: extract apply_components_by_name generic helper (single-pass filter_map replacing separate filter+map passes) - override_apply: extract check_unknown_entity_names generic helper replacing 5 near-identical check_unknown_*_names functions - override_apply: extract apply_*_components helpers per entity type - override_apply: fix entity-level Set_version from no-op to actually setting the entity version field before applying component overrides - override_apply: fix = Some Ignore structural equality → exhaustive pattern matching on entity action in all 5 apply_*_overrides - gir_gen: fix List.length > 0 → <> [] (O(1) check) - gir_gen: add comment explaining ~functions:[] is correct - overrides.md: document entity-level version behaviour (Design Decision #2), mark Phase 2 tasks complete with implementation notes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tial override files - Add `gir_gen overrides` subcommand to extract Since version annotations from GIR doc text into override sexp files; also reads version XML attribute directly from <member>/<flag>/<field> elements (fixing GDK and other namespaces that use XML attrs rather than doc text) - Add override_extractor.ml with extract_since_version (Re regex) for testability; used by generate_overrides in gir_gen.ml - Fix gir_parser.ml: member_version, flag_version, field_version were hardcoded None — now read from the element's version XML attribute - Fix override_apply.ml: warn_unknown_components checked the filtered (post-ignore) entity list causing false-positive warnings for successfully-ignored items; now checks the original entity list - Add parse_doc_text helper in gir_parser.ml that skips nested XML elements instead of failing, enabling robust doc text extraction - Add test_override_extractor.ml with 10 tests (unit + integration) - Update generate-bindings.sh to pass -o override file to all 9 references and generate commands - Add overrides/ directory with initial sexp files for all 9 namespaces: gtk.sexp migrated from hardcoded exclude lists; gdkpixbuf/graphene/ gio/pango populated with Since version entries Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…or doc-text-only The GIR parser now reads the `version` XML attribute from <member>, <flag>, and <field> elements into member_version/flag_version/field_version fields, so code generation can use them natively without override files. The `gir_gen overrides` extractor remains doc-text-only (Since X.Y patterns), since XML version attributes are already captured by the parser and don't need to be duplicated into override sexp files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mplementation - Task 1.1b: correct the claim that GIR XML never includes version attrs on member/field elements — some GDK4/GTK4 GIR files do use version="4.x" - Note that gir_parser.ml now reads version XML attrs into member_version/ flag_version/field_version natively - Phase 3 extractor description: clarify it reads only <doc> text, not XML version attrs (which the parser already handles) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- override_extractor.ml: fix docstring — regex matches literal 'Since' (capital S, no prefix), not 'Available since' or lowercase variants; clarify this explicitly - test_override_extractor.ml: add record_field_doc integration test covering the parse_doc_text call site in parse_record_contents (was not previously tested) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ride files - exclude_list.ml: remove variadic_function_exclude_list, is_variadic_function, platform_specific_type_exclude_list, is_platform_specific_type, type_name_exclude_list, is_excluded_type_name, function_exclude_list, is_excluded_function, should_skip_class — all migrated to override sexp files - gir_parser.ml: remove is_platform_specific_type guards from parse_enumeration and parse_bitfield — previously-skipped enums/bitfields now parsed and filtered by the override system - filtering.ml: remove is_excluded_type_name check from has_simple_type; remove property_exclude_list and its check from should_generate_property; remove method_has_excluded_type; remove is_excluded_function and is_variadic_function (from Exclude_list) from should_skip_method_binding; remove banned_records from should_skip_private_record; simplify should_generate_class and should_generate_interface - library_module.ml: remove should_skip_class filter (overrides handle this) - layer1_property.ml: remove is_excluded_type_name check from has_property_type_mapping - Override files updated: gdkpixbuf.sexp (PixbufNonAnim, PixbufModule, PixbufModulePattern), gio.sexp (SettingsBackend), gsk.sexp (BroadwayRenderer, NglRenderer) — each ignored in their correct namespace - Deleted stale generated files for now-ignored types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Types that existed on main and compiled fine were incorrectly excluded. The exclusions were premature — the old hardcoded lists weren't fully preventing generation of these types (stale files remained in the repo). Un-ignored (now generated): - GTK: PageSetup, PrintContext, PrintOperation, PrintSettings, PrintDialog, PrintOperationPreview (interface) Kept ignored (genuinely not in public headers / require special setup): - GTK: PrintBackend (GtkPrintBackend not in public headers), PrintJob, PrintUnixDialog, PageSetupUnixDialog, Printer, PrintCapabilities (bitfield), License (enum) - GIO: SettingsBackend (requires G_SETTINGS_ENABLE_BACKEND define) - GSK: BroadwayRenderer, NglRenderer (not in standard public headers) - GdkPixbuf: PixbufNonAnim, PixbufModule, PixbufModulePattern (GdkPixbufModule/Pattern not in public gdk-pixbuf.h) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Files created during intermediate regeneration passes (before ignore entries were finalised) left unreferenced stale files. Remove them: - gdkpixbuf: gPixbuf_module/pattern/non_anim, pixbuf_module/pattern/non_anim, ml_pixbuf_module/pattern/non_anim_gen.c - gio: gSettings_backend, settings_backend, ml_settings_backend_gen.c - gsk: gBroadway_renderer, broadway_renderer, gNgl_renderer, ngl_renderer, ml_broadway_renderer/ngl_renderer_gen.c Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Generalise the five per-entity apply functions into a single apply_entity_overrides helper, eliminating ~100 lines of repetitive code. Also replace the temp-file round-trip in parse_overrides_from_string with a direct Sexp.of_string call. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add [@@deriving sexp] to override_types.ml/.mli (previously only eq). Add 9 round-trip tests: serialise library_overrides with sexplib, deserialise back, verify structural equality. Covers all entity types. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add test_override_e2e.ml: parses a synthetic GIR XML, applies programmatic overrides (ignore Button class, ignore Widget.create method), and asserts on the apply_overrides result. Verifies ignored_entities list, method filtering, and absence of spurious warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…notes Document the apply_entity_overrides generic helper refactor and the parse_overrides_from_string cleanup. Add implementation note to Task 5.1 about [@@deriving sexp] being deferred from Phase 1 and the round-trip using sexplib serialisation (not the human-format parser). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tfield converters Add emit_member_branch helper to enum_code.ml. Update generate_c_enum_converters and generate_c_bitfield_converters to accept ~class_version and emit #if/#else caml_failwith/#endif guards around individual members/flags when member_version is set and newer than the class version. Update generate_enum_files in gir_gen.ml to pass ~class_version to both converter functions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… in enum converters Add test_enum_member_version.ml: 9 tests covering no-guard baseline, member-version-only guard (with #else caml_failwith), class-guard suppression (member_version <= class_version), member-guard emission (member_version > class_version), and bitfield equivalents. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…, scripts/README, README, CLAUDE - README_GIR_GEN.md: add `overrides` as a third command, document -o/--overrides flag for generate and references commands, add Override System section with format reference, workflow, and examples - scripts/README.md: rewrite to cover all 9 namespaces, describe two-phase generation with -o flag, add Override files section - README.md: add link to overrides/ directory in Key Documents - CLAUDE.md: update single-library regeneration example to include -o flag, add note about override files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nto manual-overrides
- docs/plans/overrides.md: mark all phases complete (1-7); replace Phase 7 task stubs with what was actually implemented - architecture/README.md: update pipeline diagram to show override_parser/apply stage; add override modules to Core Modules table; add Override System section with design summary; fix cross-namespace reference file paths - architecture/gir_gen/overrides.md: new doc covering pipeline position, type model, parser decisions, apply logic, version guard behaviour, and update workflow Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ge safety - render_version_component: add ~kind parameter, emit (member/field NAME (version X.Y)) instead of bare (NAME (version X.Y)) — the parser requires the kind keyword as the first sexp atom to match parse_components_of_kind - Fix gio.sexp, pango.sexp, graphene.sexp, gdkpixbuf.sexp: convert 250+ bare-name member entries to (member NAME ...) / (field NAME ...) form - generate_overrides: merge with existing file instead of overwriting — preserves all (ignore) entries, replaces only version annotations with fresh GIR data - Regenerate ml_gio_enums_gen.c, ml_pango_enums_gen.c, ml_graphene_enums_gen.c, ml_gdkpixbuf_enums_gen.c with correct per-member version guards Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ries) Previously parse_components_of_kind silently skipped any sexp whose first atom did not match the expected kind — causing bare-name entries like (foo (version "X.Y")) to be silently discarded instead of reported. Add validate_body_elements which runs once per entity body and fails with Unknown_component_kind on any list-form element whose leading atom is not a valid kind for that entity type (or the action markers ignore/version). Valid kinds per entity: - class: constructor, method, property, signal - interface: method, property, signal - record: field, constructor, method, function - enumeration: member, function - bitfield: member Add Unknown_component_kind constructor to parse_error (ml + mli) and four new parser tests covering: bare-name member in enum, bare-name member in bitfield, wrong kind (field) in class, wrong kind (signal) in record. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements a declarative s-expression override system for the GIR code generator, replacing all hardcoded exclusion lists scattered across
exclude_list.ml,filtering.ml, andgir_parser.mlwith per-namespace configuration files inocgtk/overrides/.override_types.ml) and s-expression parser (override_parser.ml) with full unit test coverageoverride_apply.ml) integrated into thegir_gen generateandgir_gen referencespipelines; version overrides feed directly into C#ifversion guardsgir_gen overridessubcommand extracts "Since X.Y" version annotations from GIR<doc>text;generate-bindings.shupdated to wire-ooverride files into all 9 namespace generations; initialocgtk/overrides/files created; GIR parser extended to readversionXML attributes on<member>/<field>elements nativelySettingsBackend→gio.sexp,BroadwayRenderer/NglRenderer→gsk.sexp,PixbufNonAnim/PixbufModule/PixbufModulePattern→gdkpixbuf.sexp, print-related GTK types stay ingtk.sexp)[@@deriving sexp]added to override types; 9 round-trip sexp tests; end-to-end smoke test parsing synthetic GIR XML, applying overrides, and asserting on results#if/#elseversion guards in enum/bitfield C converters and_decls.hdeclaration headers; 9 unit testsREADME_GIR_GEN.mddocuments theoverridescommand,-oflag, and override file format;scripts/README.mdupdated for all 9 namespaces and override workflow;README.mdandCLAUDE.mdupdated with override file references; architecture docs inocgtk/architecture/updated with pipeline diagram and override system designrender_version_componentwas emitting(NAME (version "X.Y"))but the parser'sparse_components_of_kindrequires(member NAME ...)/(field NAME ...)as the first atom — all member version entries were silently skipped, producing no version guards. Fixed by adding a~kindparameter to the render function.gio.sexp,pango.sexp,graphene.sexp,gdkpixbuf.sexpcorrected to(member NAME ...)/(field NAME ...)formgir_gen overridesnow reads any existing override file before writing, preserving all(ignore)entries and only replacing version annotations with fresh GIR data — re-running the command no longer loses manual editsml_gio_enums_gen.c,ml_pango_enums_gen.c,ml_graphene_enums_gen.c,ml_gdkpixbuf_enums_gen.cupdated with correct per-member version guardsTest plan
cd ocgtk && dune build— clean buildcd ocgtk && xvfb-run dune runtest— all tests passbash scripts/generate-bindings.sh— no warnings, all 9 namespaces generate cleanlySettingsBackend,BroadwayRenderer,PixbufModule, etc.) no longer appear in generated outputml_gsk_enums_gen.candgsk_decls.h(e.g.#if GTK_CHECK_VERSION(4,14,0)aroundGskFillRule)ml_gio_enums_gen.c(168 guards),ml_pango_enums_gen.c,ml_graphene_enums_gen.c,ml_gdkpixbuf_enums_gen.c🤖 Generated with Claude Code