Zig 0.15.2 upgrade with translate-c and nix fixes#106
Closed
jhf wants to merge 25 commits into
Closed
Conversation
This commit updates the codebase for compatibility with Zig 0.15.1,
addressing several breaking changes. A new build step for examples is
also introduced.
Key changes include:
- `usingnamespace` Removal: All instances of `usingnamespace` have been
replaced with explicit namespacing. C imports are now scoped within a
`pg` struct, improving code clarity across the project.
- Standard Library API Updates:
- `std.ArrayList`: Switched to the new unmanaged-by-default API, using
`std.array_list.Managed` where an allocator is needed.
- I/O ("Writergate"): Updated file writing logic to use the new
buffered I/O interfaces, including `flush()` calls. Formatted
string allocation now uses `allocPrintSentinel`.
- Build System:
- `build.zig` now uses `b.addModule` instead of the deprecated
`b.createModule`.
- A new `examples` step is added to conveniently build all example
extensions via `zig build examples`.
- Language Syntax:
- `callconv(.C)` updated to the new `callconv(.c)`.
- `@call(.no_async, ...)` updated to `@call(.auto, ...)` after changes
to async handling.
This commit updates all examples to be compatible with the breaking
changes introduced in Zig 0.15.1. This aligns the examples with the
updates recently made to the core `pgzx` library.
Specific changes include:
- Build System: Updated `build.zig` files to use `source_file`
instead of the deprecated `root_source_file` in `pgzx.Project`.
- Standard Library: Migrated from `std.ArrayList` to the new
`std.array_list.Managed`, and replaced `std.fmt.allocPrintZ` with
`std.fmt.allocPrintSentinel`.
- Formatting: Adjusted JSON serialization in `pgaudit_zig` to use the
new `{f}` format specifier with `std.json.fmt`.
- Namespacing: Corrected C imports to use `pgzx.c.pg` for consistency.
- Syntax: Updated `callconv(.C)` to the new `callconv(.c)`.
See include/postgresql/server/fmgr.h for the new abi_fields part of the struct.
In both zig build for 0.14 and 0.15.1 as found in release notes.
In reality a NOOP but for consistency and a single source of truth.
Such as with `zig build unit -Dtest_filter=pgzx.collections.htab.TestSuite_HTab` to debug only that test.
- Both PG17 and PG18 from Postgres.app crash when calling hash_get_num_entries() - Verified with pure C extension - not a Zig issue - Added zero-initialization for HASHCTL struct (correct PostgreSQL practice) - All other tests (43) pass
- Add compile-time detection of Pg_magic_struct format (PG18+ has nested abi_fields, PG16 and earlier have flat fields) - Update htab test disable comment with more accurate findings: direct pg.hash_search() calls work, wrapper methods crash on arm64
- Upgrade zig-stable from 0.14.0 to 0.15.2 in flake.nix - Remove broken zls dependency (install separately if needed) - Pin pre-commit-hooks-nix to compatible version - Fix addRegress and addRunTests to use actual pkglibdir from pg_config instead of hardcoded lib/postgresql path - Auto-configure dynamic_library_path in pginit for nix postgres - Update htab test disable comment
- Add parameterized devshell.nix that accepts postgresql package - Add pg16, pg17, pg18 dev shells to flake.nix - Update nixpkgs to unstable for PG17/18 support - Add ci/matrix-test.sh for testing against multiple PG versions Note: PG16 tests pass. PG17/18 require nix cache population.
- Add -f flag to ln commands to force overwrite existing symlinks - Make existing files writable before copying to avoid permission errors - Use ln -sfn for the default symlink to handle directory symlinks
- Use more reliable process matching for killing postgres - Add lsof-based port cleanup as fallback - Increase sleep time to ensure port is released
When pg_config points to a nix store path, the test runner was trying to load extensions from the nix store instead of the install prefix. Always use install_prefix/lib for the test extension path.
- Use nixpkgs 24.11 for stable PG16/17 support - Set PG_CONFIG to point to nix dev package for correct headers - Disable pre-commit (broken Swift dependency in nixpkgs 24.11) - Remove pre-commit hook from shell since it's disabled
- Add nixpkgs-unstable input for PG18 (not available in 24.11) - Update devshell.nix to detect pg_config location (different in unstable) - Fix pglocal to use PG_CONFIG env var when set - PG16/17 from nixpkgs 24.11, PG18 from nixpkgs-unstable pre-commit issue details: - pre-commit package requires building Swift and .NET from source - These are massive builds (hours) that often fail on macOS - Keeping pre-commit disabled; linters still available individually
On macOS, Zig's linker uses two-level namespace by default, which binds hash_create, hash_destroy, and hash_search to BSD hsearch functions in libSystem instead of PostgreSQL's internal functions. This causes crashes when using pgzx.HTab. Root cause: macOS libSystem exports hash_create/hash_destroy with different signatures than PostgreSQL. Zig's MachO linker doesn't support -flat_namespace or -bundle linking which would fix the symbol resolution. Changes: - Add detailed explanation to AGENTS.md Gotchas section - Update src/testing.zig with clear comment on why HTab tests are disabled - Add C test cases demonstrating the issue and workaround: - test_twolevel.c: demonstrates bundle_loader fix - test_func_addr.c: shows symbol address mismatch - test_hsearch_v2.c: simple reproduction case - test_hsearch.c: original comprehensive test
Replace the deprecated @cImport approach with build-time translate-c for C header translation. This aligns with Zig's direction of removing @cImport in future versions. Changes: - Add src/pgzx/c/include/headers.h with all PostgreSQL includes - Update build.zig to use addTranslateC() instead of @cImport - Rename module from 'pgzx_pgsys' to 'c_translated' - Update all internal imports to use the new module name - Keep test_filter support for selective test running This change is based on PR xataio#105 by @mcadariu, adapted to work with our existing infrastructure and test framework. Co-authored-by: Marius Cadariu <mcadariu@users.noreply.github.com>
The zig build docs command destructively replaces the docs/ folder, which would delete any manually added files.
Author
|
I did also use AI tools to create https://codeberg.org/veridit/zig/src/branch/feature/macho-bundle-loader that adds the requires support to Zig. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR upgrades pgzx to Zig 0.15.2 with comprehensive fixes:
@cImportto build-timetranslate-c- Aligns with Zig's direction of deprecating@cImportHTab Crash Root Cause
The HTab tests crash on macOS due to symbol collision with BSD hsearch. macOS's
libSystemexportshash_create,hash_destroy, andhash_searchwith different signatures than PostgreSQL's internal functions. Zig's linker uses two-level namespace by default, which binds these symbols to libSystem instead of PostgreSQL.There is no Zig workaround - the linker doesn't support
-flat_namespaceor-bundlelinking. A Zig issue draft is included atdocs/zig-macos-flat-namespace-issue.md.HTab tests are disabled until Zig adds flat namespace support.
Changes
Build System
@cImportwithaddTranslateC()in build.zigsrc/pgzx/c/include/headers.hfor C header translationpgzx_pgsystoc_translatedNix Infrastructure
Documentation
Testing
All 43 unit tests pass (with HTab disabled):
Credits
This PR builds on work from PR #105 by @mcadariu, specifically:
Co-authored-by: Marius Cadariu mcadariu@users.noreply.github.com