Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite autowrap #5

Merged
merged 25 commits into from
Jul 11, 2024
Merged

Rewrite autowrap #5

merged 25 commits into from
Jul 11, 2024

Conversation

kylewlacy
Copy link
Member

This PR is a massive overhaul to the autowrap functionality in this repo.

From a high level, the main difference is in the brioche-packer autowrap subcommand. Previously, this took a lot of command-line flags to control autowrapping. This has been changed to take the config as a JSON object, along with variable values to set paths (the variables are important as to avoid issues that could come up when interpolating paths in the JSON config).

There are also some new capabilities, like autowrapping shebang scripts, using separate options for wrapping dynamic binaries and libraries, and rewrapping already-wrapped programs.

Here's an example invocation:

brioche-packer autowrap \
    "$BRIOCHE_OUTPUT" \
    --var "toolchain=path:${toolchain}" \
    --var "dynamicBinaryPackedExecutable=path:${dynamicBinaryPackedExecutable}" \
    --config '
      {
        "paths": ["lib/libfoo.so", "lib/libbar.so"],
        "linkDependencies": [{variable: "toolchain"}],
        "selfDependency": true,
        "dynamicBinary": {
          "packedExecutable": {variable: "dynamicBinaryPackedExecutable"},
        },
        "rewrap": {},
      }
    '

The positional argument (RECIPE_PATH) specifies the root path to wrap (this is used for resolving some relative paths and is the base path used for glob patterns). Each variable takes the format --var [name]=[type]:[value], with the only type being path currently. Passing --schema will also print the JSON schema of the config (although currently this doesn't include documentation for each option).

  • paths: Set to a list of paths to wrap explicitly. This will fail if any path cannot be wrapped. Mutually exclusive with globs
  • globs: Set to a list of glob patterns to try to wrap. If any matched file cannot be wrapped, it will be ignored. Mutually exclusive with paths
  • linkDependencies: A list of paths to link against. This is used for finding dynamic libraries, the dynamic interpreter, and programs from shebang scripts
  • selfDependency: Include the recipe itself in the list of linkDependencies. This is useful if a package includes libraries that are referenced by its own binaries
  • dynamicBinary: Config used when wrapping a dynamic binary. If unset, dynamic binaries will be skipped
    • packedExecutable (required): Path to the packed executable to use when wrapping the binary. This will usually be the path to the brioche-packed-userland-exec executable from this crate
    • extraRuntimeLibraryPaths: Extra paths to search for libraries at runtime when the executable is run. Paths are relative to the recipe dir
    • libraryPaths: Extra paths to search for libraries at link time
    • skipLibraries: A list of library names to not link if if included in the DT_NEEDED section
    • extraLibraries: A list of library names to link even if not included in the DT_NEEDED section
    • skipUnknownLibraries: When set to true, skip any libraries that could not be found instead of erroring out
  • sharedLibrary: Config used when wrapping a shared library. If unset, shared libraries will be skipped
    • libraryPaths: Extra paths to search for libraries at link time
    • skipLibraries: A list of library names to not link if if included in the DT_NEEDED section
    • extraLibraries: A list of library names to link even if not included in the DT_NEEDED section
    • skipUnknownLibraries: When set to true, skip any libraries that could not be found instead of erroring out
  • script: Config used when wrapping a shebang script. If unset, shebang scripts will be skipped
    • packedExecutable (required): Path to the packed executable to use when wrapping the script. This will usually be the path to the brioche-packed-plain-exec executable from this crate
    • env: An object representing env vars to change when the script gets run. Each key is an env var name, and each value with one of the following types:
      • clear: Clear the env var even if it is already set
      • inherit: Inherit the env var even if clearEnv is enabled
      • set: Set the env var to the provided value
      • fallback: Set the env var if is empty
      • prepend: Prepend to the current env var (separated with the provided separator). If the env var is empty, the separator will not be included
      • append: Append to the current env var (separated with the provided separator). If the env var is empty, the separator will not be included
    • clearEnv: Start the process without any env vars set, except those explicitly set in env
  • rewrap: Config used when encountering a dynamic binary, shared library, or script that was already wrapped. If unset, already-wrapped things will be left as-is
    • (There are currently no extra options for rewrapping, so rewrapping is enabled by setting this field to an empty object)

kylewlacy added 25 commits June 29, 2024 16:20
There were a few tweaks to work better with `brioche-ld` as well
Each file in the search path will be a candidate, and will be checked by
both filname and by the `DT_SONAME` field
This fixes a regression seen when building `as` from `binutils`. Here's
what went wrong:

1. `as` was built and linked against `libbfd-2.41.so`
2. `libbfd-2.41.so` was passed as an input directly via a symlink called
   `libbfd.so`
3. Because the library has a `DT_SONAME` field, the resulting binary has
   a `NEEDED` field for `libbfd-2.41.so`
4. Since we were using the direct filename from the input rather than the
   library name, `libbfd.so` was added as a resource to the packed binary
5. When ran, `as` was looking for `libbfd-2.41.so` instead of
   `libbfd.so`, so it failed to run

The issue is in step (4). Now, the library is added by whatever's in the
`NEEDED` list instead of what the input filename happened to be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant