Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ad67513
docs(plans): Plan for overriding GIR ignore list and versions from a …
chris-armstrong Apr 4, 2026
409c150
Merge branch 'main' of github.com:chris-armstrong/ocgtk into manual-o…
chris-armstrong Apr 4, 2026
2cb8512
feat(overrides): Phase 1 — override types and s-expression parser
chris-armstrong Apr 4, 2026
2789aaf
feat(overrides): Phase 2 — override application and pipeline integration
chris-armstrong Apr 4, 2026
55951dd
refactor(overrides): fix Phase 2 code review issues
chris-armstrong Apr 4, 2026
e10b96d
feat(overrides): Phase 3 — extract versions from GIR and generate ini…
chris-armstrong Apr 4, 2026
0d6a87d
fix(overrides): parse version XML attr in gir_parser but keep extract…
chris-armstrong Apr 4, 2026
d95167b
docs(overrides): update plan to reflect actual member version field i…
chris-armstrong Apr 4, 2026
8308204
fix(overrides): address reviewer feedback on Phase 3
chris-armstrong Apr 4, 2026
287fe8e
feat(overrides): Phase 4 — remove hardcoded exclusion lists, use over…
chris-armstrong Apr 4, 2026
f4dffd3
fix(overrides): restore generation of GTK print classes and other types
chris-armstrong Apr 4, 2026
9f797af
fix(overrides): delete stale generated files for ignored types
chris-armstrong Apr 4, 2026
608b280
refactor(overrides): simplify apply and parser
chris-armstrong Apr 4, 2026
2f60475
feat(overrides): Phase 5.1 — round-trip sexp tests for override types
chris-armstrong Apr 4, 2026
1af6d8b
feat(overrides): Phase 5.2 — end-to-end smoke test for override pipeline
chris-armstrong Apr 4, 2026
2b55b80
docs(overrides): update plan to reflect Phase 2 refactor and Phase 5 …
chris-armstrong Apr 4, 2026
6d83e14
feat(overrides): Phase 6.1-6.4 — per-member version guards in enum/bi…
chris-armstrong Apr 5, 2026
c495931
feat(overrides): Phase 6.6 — unit tests for per-member version guards…
chris-armstrong Apr 5, 2026
f8b3f4c
docs(overrides): Phase 7 — document override system in README_GIR_GEN…
chris-armstrong Apr 5, 2026
1415fe7
Merge branch 'manual-overrides' of github.com:chris-armstrong/ocgtk i…
chris-armstrong Apr 5, 2026
d4e29b9
docs(overrides): update plan to complete, refresh architecture docs
chris-armstrong Apr 5, 2026
56be70c
chore(code): regenerate with enumeration and bitfield version guards
chris-armstrong Apr 5, 2026
d5adc0e
fix(overrides): render_version_component missing kind prefix; add mer…
chris-armstrong Apr 5, 2026
bf590b1
chore(claude): remove settings json file
chris-armstrong Apr 5, 2026
0637024
fix(override_parser): error on unknown component kinds (bare-name ent…
chris-armstrong Apr 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,6 @@ __pycache__/
.context/logs/
.context/.scratchpad.key
.claude/settings.local.json
ocgtk/.claude/settings.local.json
.claude/worktrees/
ocgtk/.claude/worktrees/
8 changes: 6 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,16 +142,20 @@ For generating GTK bindings from GObject Introspection (GIR) files:
bash scripts/generate-bindings.sh
```

This builds the generator, generates reference files for all 9 namespaces, then generates bindings with correct cross-namespace dependencies.
This builds the generator, generates reference files for all 9 namespaces, then generates bindings with correct cross-namespace dependencies. Override files in `ocgtk/overrides/` are applied automatically (one per namespace).

**To regenerate a single library manually:**
```bash
cd ocgtk
dune exec src/tools/gir_gen/gir_gen.exe -- generate /usr/share/gir-1.0/Gtk-4.0.gir src/gtk
dune exec src/tools/gir_gen/gir_gen.exe -- generate \
-o overrides/gtk.sexp \
/usr/share/gir-1.0/Gtk-4.0.gir src/gtk
```

NOTE: For other libraries, use `src/<short_name>`. For example, src/pango for Pango, src/gsk for GSK, src/gdk for GDK, etc.

**Override files** (`ocgtk/overrides/<ns>.sexp`) control which entities are ignored during generation and set version guards on enum/bitfield members. Pass `-o overrides/<ns>.sexp` to `generate` or `references`. See [README_GIR_GEN.md — Override System](ocgtk/src/tools/README_GIR_GEN.md#override-system).

**⚠️ IMPORTANT:** Use `src/gtk` NOT `src/gtk/generated` as the output directory. The generator automatically creates the `generated/` subdirectory. Using `src/gtk/generated` will create a nested `src/gtk/generated/generated/` directory.


Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ This library is distributed under the terms of the GNU Library General Public Li
- **Development Setup**: [SETUP.md](SETUP.md)
- **Security Guidelines**: [SECURITY_GUIDELINES.md](SECURITY_GUIDELINES.md)
- **GIR Code Generation**: [ocgtk/src/tools/README_GIR_GEN.md](ocgtk/src/tools/README_GIR_GEN.md)
- **GIR Override Files**: [ocgtk/overrides/](ocgtk/overrides/) — per-namespace sexp files controlling entity ignores and version guards

## Resources

Expand Down
33 changes: 0 additions & 33 deletions ocgtk/.claude/settings.local.json

This file was deleted.

72 changes: 51 additions & 21 deletions ocgtk/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,25 +34,30 @@ Generated code lives in `src/<namespace>/generated/`:

## gir_gen Architecture

### Pipeline (4 Layers)
### Pipeline

```
GIR XML
├─► parse/gir_parser.ml ──► types.ml (AST)
│ │
│ ▼
│ type_mappings.ml
│ │
├──────────────────────────────┼──────────────────────────────┐
▼ ▼ ▼
generate/ generate/ generate/
c_stubs.ml layer1/*.ml class_gen*.ml
(Layer 0: C FFI) (Layer 1: Low-level ML) (Layer 2: High-level)
│ │ │
▼ ▼ ▼
ml_*_gen.c widget.mli/ml gWidget.ml
C stubs External functions OCaml classes
GIR XML overrides/<ns>.sexp
│ │
▼ ▼
parse/gir_parser.ml override_parser.ml
│ │
└──────── AST ───────┘
override_apply.ml (filter ignored entities, set versions)
type_mappings.ml (build generation context)
┌─────────────┼──────────────────┐
▼ ▼ ▼
generate/ generate/ generate/
c_stubs.ml layer1/*.ml class_gen*.ml
(Layer 0: C) (Layer 1: ML) (Layer 2: ML)
│ │ │
▼ ▼ ▼
ml_*_gen.c widget.mli/ml gWidget.ml
C stubs External decls OCaml classes
```

**Layer 3** (Signals): `generate/signal_gen.ml` produces `gWidget_signals` classes.
Expand All @@ -62,11 +67,14 @@ C stubs External functions OCaml classes
| Module | Purpose |
|--------|---------|
| `types.ml` | AST for GIR elements (classes, methods, enums, records) |
| `parse/gir_parser.ml` | XML parsing → AST |
| `parse/gir_parser.ml` | XML parsing → AST; reads `version` XML attrs on members/fields |
| `type_mappings.ml` | C→OCaml type mapping (`gint`→`int`, etc.) + cross-namespace resolution |
| `override_types.ml` | Override type definitions (ignore, version actions) |
| `override_parser.ml` | S-expression parser for `overrides/<ns>.sexp` files |
| `override_apply.ml` | Apply overrides to AST before ctx build (filter + version) |
| `dependency_analysis.ml` | Tarjan SCC for cyclic dependency handling |
| `filtering.ml` | Method/property filtering (out params, unknown types) |
| `exclude_list.ml` | Platform-specific skips, variadic functions |
| `filtering.ml` | Method/property filtering (out params, unknown types, varargs) |
| `exclude_list.ml` | Residual structural skips (`*Private` records, etc.) |

### Generation Modules

Expand Down Expand Up @@ -106,12 +114,34 @@ Cyclic dependencies (e.g., `Application` ↔ `Window`) are combined into single
- Generates `application_and__window_and__window_group.ml`
- Type references: simple names within cycle (`Window.t`), qualified across cycles

### Override System

Per-namespace override files (`ocgtk/overrides/<ns>.sexp`) configure what gets
generated without modifying the generator source. Applied in `override_apply.ml`
**before** the type-mapping context is built — ignored entities are absent from
`ctx`, so methods referencing them are naturally skipped by the existing
unknown-type filter in `filtering.ml`.

Two actions are supported:
- `(ignore)` — remove the entity/component entirely from all generation stages
- `(version "X.Y")` — set a version field, which feeds into `#if` C version guards

Version guards appear at two granularities:
- **Entity-level**: wraps the entire C converter/stub (`#if MACRO(X,Y,0)`)
- **Member-level**: wraps individual `case`/`else if` branches inside a converter

See [`gir_gen/overrides.md`](gir_gen/overrides.md) for the design rationale and
[`README_GIR_GEN.md`](../src/tools/README_GIR_GEN.md#override-system) for the file format.

### Cross-Namespace Types

Resolved via reference files (`<ns>_refs.txt`):
Resolved via reference files (`_build/references/<ns>-references.sexp`):
- Classes/records: `Ocgtk_<ns>.<Ns>.Wrappers.<Module>.t`
- Enums/bitfields: `Ocgtk_<ns>.<Ns>.<enum_name>`

The `gir_gen references` command also accepts `-o <ns>.sexp` so ignored entities
are excluded from reference output.

## Key Documentation

- **[README_GIR_GEN.md](../src/tools/README_GIR_GEN.md)** - Complete generator usage and status
Expand Down
153 changes: 153 additions & 0 deletions ocgtk/architecture/gir_gen/overrides.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# GIR Override System Architecture

The override system lets per-namespace sexp files control what the generator emits
without touching generator source. It replaces the former hardcoded exclusion lists
in `exclude_list.ml`, `filtering.ml`, and `gir_parser.ml`.

## Files

```
ocgtk/overrides/ # One file per namespace, committed to the repo
cairo.sexp
gio.sexp
gdk.sexp
graphene.sexp
gdkpixbuf.sexp
pango.sexp
pangocairo.sexp
gsk.sexp
gtk.sexp

src/tools/gir_gen/
override_types.ml/.mli # Override type definitions
override_parser.ml/.mli # S-expression parser
override_apply.ml/.mli # Apply overrides to parsed GIR data
```

## Pipeline Position

Override application happens **after GIR parsing, before ctx build**:

```
gir_parser.ml → raw AST → override_apply.ml → filtered AST → type_mappings ctx → generators
```

This ordering is required. Ignored entities must be absent from `ctx` so that
`find_type_mapping_for_gir_type` returns `None` for their types, causing any methods
that reference them to be silently skipped by the existing unknown-type filter in
`filtering.ml`. If overrides were applied after ctx build, ignored types would still
appear in the context and their referencing methods would be generated.

## Type Model (`override_types.ml`)

```ocaml
type override_action = Ignore | Set_version of string

type component_override = { component_name : string; action : override_action }

type class_override = {
class_name : string;
class_action : override_action option;
constructors : component_override list;
methods : component_override list;
properties : component_override list;
signals : component_override list;
}
(* interface_override, record_override, enum_override, bitfield_override: similar *)

type library_overrides = {
library_name : string;
classes : class_override list;
interfaces : interface_override list;
records : record_override list;
enums : enum_override list;
bitfields : bitfield_override list;
functions : component_override list;
}
```

`[@@deriving eq]` on all types enables structural equality for tests.
`[@@deriving sexp]` is also derived but is **not** used for the human-authored file
format — the derived sexp mirrors OCaml record structure verbatim and would be
unreadable. The hand-written parser in `override_parser.ml` is the authoritative
parser. The derived sexp is used only for round-trip tests.

## Parser (`override_parser.ml`)

Reads the human-friendly format:

```sexp
(overrides
(library "Gtk")
(class Widget
(method foo (ignore))
(property bar (version "4.14"))
)
(enumeration StateFlags
(member new_flag (version "4.16"))
)
)
```

Key parser decisions:
- **Hard error on malformed sexp**: halts generation immediately
- **Hard error on duplicate entities/components**: two `(class Widget ...)` in the
same file is always rejected
- **Warning on unknown names**: override references an entity or component not in the
GIR data — generation continues, warning is printed (catches typos)
- **`(function ...)` is context-disambiguated**: at the top level of `(overrides ...)`
it's a namespace-level function; nested inside `(record ...)` or `(enumeration ...)`
it's an entity-level function. No extra keyword needed.
- **Bitfield members use `member` keyword** (same as enum members), stored in
`bitfield_override.flags`

## Apply (`override_apply.ml`)

`apply_overrides` processes entities in a single pass:

1. **Entity-level ignore** (`class_action = Some Ignore`): removes the entity from
all lists. Silently drops any component overrides on the same entity.
2. **Entity-level version** (`class_action = Some (Set_version v)`): sets the version
field on the surviving entity, then processes component overrides.
3. **Component-level ignore**: removes the method/property/etc. from the entity's list.
4. **Component-level version**: updates `member_version`/`flag_version`/`field_version`
on the surviving component.

Implementation uses a single generic `apply_entity_overrides` helper to avoid the
5-way copy-paste that existed in the original draft.

`apply_result` contains:
- Filtered entity lists
- `ignored_entities : string list` — names of removed entities (for logging)
- `warnings : string list` — unknown entity/component name warnings

## Version Guards

Member versions feed into C code generation in `enum_code.ml`:

- **Enum converters**: each `case` in the C-to-OCaml switch and each `else if` branch
in the OCaml-to-C chain can be individually wrapped in `#if MACRO(X,Y,0)`.
- **Bitfield converters**: same for `if (flags & ...)` branches.
- **`_decls.h` headers**: converter declarations in `<ns>_decls.h` are also guarded
so dependent namespaces don't see symbols that don't exist at their compile target.

The `emit_member_branch` helper in `enum_code.ml` handles the `#if`/`#else caml_failwith`/`#endif`
pattern. The `#else` branch calls `caml_failwith` so that OCaml code passing a version-guarded
variant to an older library fails at runtime with a clear message rather than silently hitting
the unknown-value default handler.

Member version vs. class version:
- If `member_version <= class_version`, the member guard is redundant (the outer class
guard already covers it) and is omitted.
- If `member_version > class_version` (or class has no version), the member guard is
emitted as an inner `#if` inside the converter.

## Updating Override Files

Override files are generated once with `gir_gen overrides` (extracts `Since X.Y`
from GIR `<doc>` text), then hand-edited to add `(ignore)` entries. When GTK
upgrades, re-run `gir_gen overrides` and merge the diff.

The recommended workflow is `bash scripts/generate-bindings.sh` from the repository
root, which wires `-o overrides/<ns>.sexp` into all 9 `generate` and `references`
invocations automatically.
Loading
Loading