Unable to install formula with "outdated pinned dependency"

[NickLaMuro]

Please note that we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

• are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
• ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
• ran brew update and can still reproduce the problem?
• ran brew doctor, fixed all issues and can still reproduce the problem?
• ran brew config and brew doctor and included their output with your issue?

Note: The issues reported by brew doctor seemed benign and based on a bit of "puts debugging", so I don't think they are related to the issue at hand, but that output and brew config are provided below for posterity.

brew doctor output

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have the following deprecated, official taps tapped:
  Homebrew/homebrew-dupes
Untap them with `brew untap`.

Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
  /usr/local/lib/libwkhtmltox.0.12.3.dylib

Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected header files:
  /usr/local/include/wkhtmltox/image.h
  /usr/local/include/wkhtmltox/pdf.h

Warning: You have uncommitted modifications to Homebrew.
If this is a surprise to you, then you should stash these modifications.
Stashing returns Homebrew to a pristine state but can be undone
should you later need to do so for some reason.
  cd /usr/local/Homebrew/Library && git stash && git clean -d -f

Warning: Homebrew's sbin was not found in your PATH but you have installed
formulae that put executables in /usr/local/sbin.
Consider setting the PATH for example like so:
  echo 'export PATH="/usr/local/sbin:$PATH"' >> ~/.bash_profile

Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
	brew install adns gnutls gts ilmbase jasper libassuan libde265 libgcrypt
               libgpg-error libheif libksba libomp libtasn1 libunistring libusb
               little-cms2 netpbm nettle npth openexr openjpeg p11-kit pinentry
               qrencode shared-mime-info unbound x265

Run `brew missing` for more details.

brew config output

HOMEBREW_VERSION: 2.1.2-1-g9934f00-dirty
ORIGIN: https://github.com/Homebrew/brew
HEAD: 9934f0086bcefc7cb645cac9d4006943648646e1
Last commit: 9 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: c7ec7b1843dd249a26e6890f487a443b976810b1
Core tap last commit: 7 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_NO_ANALYTICS_THIS_RUN: 1
CPU: octa-core 64-bit haswell
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1001
Git: 2.20.1 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_102
macOS: 10.14.4-x86_64
CLT: 10.2.1.0.1.1554506761
Xcode: N/A
CLT headers: 10.2.1.0.1.1554506761

What you were trying to do (and why)

Install the h2o package with a pinned readline package (pinned to 8.0.0).

Extra info/rational about pinning readline (aka: "the why")

Personally, I have found that allowing readline to be unpinned will cause ruby-install/chruby to "bork" other rubies when a new ruby is installed. Details in the PR that I opened up in an attempt to address that (brew related):

postmodern/ruby-install#340

To summarize, by forcing an upgrade to all dependent packages (readline being one of them) on install of a new ruby, it will create a new directory and location for it's .dll file that ruby compiles against, and so previous versions that depended on that location of the .dll will now fail to execute functions that rely on readline such as irb (example output show in the above PR).

Pinning a package was the simplest workaround to that problem while the above PR is outstanding.

Steps below are going to recreate with sphinx-doc as it is a nested dependency from above, and makes for a smaller dependency stack to digest (also shows the issue isn't with a single formula).

Dependency "graph" for h2o (down below for illustration):

h2o
  cmake
    sphinx-doc
      python
        pkg-config
        gdbm
        openssl
        readline
        sqlite
          readline
        xz
  pkg-config
  openssl

What happened (include command output)

brew err'd out and said to "brew unpin readline" and to reinstall, as sphinx-doc required the latest version to install properly:

Command output

$ brew install sphinx-doc
Error: You must `brew unpin readline` as installing sphinx-doc requires the latest version of pinned dependencies

What you expected to happen

Install the package. readline should not require an update for this package to function.

Specifically, it seems like check that causes the above error was added here: #864

Which was solving the inverse (almost) of what I am dealing with. In the case of the reported issue it is attempting to solve, the user bumped the pinned package in question, and then it was unable to find the path necessary when installing a dependent package that depended on the pinned one.

In my case, it seems like homebrew is looking for a different version then what I have pinned, 8.0.0_1 versus 8.0.0 respectively, and so this fails the Formula#installed? check. In my case, I have 8.0.0 pinned, and to my knowledege it hasn't been upgraded, and instead is looking for 8.0.0_1 for some reason:

$ brew info readline

$ brew info readline
readline: stable 8.0.0 (bottled) [pinned at 8.0.0, keg-only]
Library for command-line editing
https://tiswww.case.edu/php/chet/readline/rltop.html
/usr/local/Cellar/readline/8.0.0 (48 files, 1.5MB)
  Poured from bottle on 2019-02-18 at 11:42:36
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/readline.rb
==> Caveats
readline is keg-only, which means it was not symlinked into /usr/local,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

==> Analytics
install: 456,582 (30 days), 1,283,263 (90 days), 4,154,455 (365 days)
install_on_request: 48,806 (30 days), 135,047 (90 days), 412,783 (365 days)
build_error: 0 (30 days)

I have some puts debugging prior to opening this issue, which I am providing the output to that below, along with the diff of the changes I made to brew:

brew install sphinx-doc (with debugging)

$ brew install sphinx-doc
Dependencies...
pkg-config
  installed?       true
    dir:           "/usr/local/opt/pkg-config"
    dir?           true
  missing_options? false
gdbm
  installed?       true
    dir:           "/usr/local/opt/gdbm"
    dir?           true
  missing_options? false
openssl
  installed?       true
    dir:           "/usr/local/opt/openssl"
    dir?           true
  missing_options? false
readline
  installed?       false
    dir:           "/usr/local/Cellar/readline/8.0.0_1"
    dir?           false
  missing_options? false
sqlite
  installed?       false
    dir:           "/usr/local/Cellar/sqlite/3.28.0"
    dir?           false
  missing_options? false
xz
  installed?       true
    dir:           "/usr/local/opt/xz"
    dir?           true
  missing_options? false
python
  installed?       false
    dir:           "/usr/local/Cellar/python/3.7.3"
    dir?           false
  missing_options? false
Error: You must `brew unpin readline` as installing sphinx-doc requires the latest version of pinned dependencies

git diff of Homebrew dir

diff --git a/Library/Homebrew/formula_installer.rb b/Library/Homebrew/formula_installer.rb
index b549b762d..a85a7d8cc 100644
--- a/Library/Homebrew/formula_installer.rb
+++ b/Library/Homebrew/formula_installer.rb
@@ -189,7 +189,13 @@ class FormulaInstaller
       EOS
     end
 
+    puts "Dependencies..."
     pinned_unsatisfied_deps = recursive_deps.select do |dep|
+      puts dep.to_formula.full_name
+      puts "  installed?       #{dep.to_formula.installed?.inspect}"
+      puts "    dir:           #{dep.to_formula.installed_prefix.to_s.inspect}"
+      puts "    dir?           #{dep.to_formula.installed_prefix.directory?.inspect}"
+      puts "  missing_options? #{(!dep.missing_options(inherited_options_for(dep)).empty?).inspect}"
       dep.to_formula.pinned? && !dep.satisfied?(inherited_options_for(dep))
     end

My guess (without knowing how the dependencies are determined prior to checking if they exist) is that brew is somehow determining that readline 8.0.0_1 is required for this install, when I already should have it satisfied with readline 8.0.0.

My work around at the moment was to just brew edit cmake and remove the depends_on "sphinx-doc" line, brew install h2o and then undo the change. Allows the reproduction steps to work, and me to continue on my way. I will attempt to figure out how Homebrew determines the dependencies in the first place some other time, but wanted to open this to make the maintainers aware of the issue.

Step-by-step reproduction instructions (by running brew commands)

I will update these steps in the future with a more accurate brew SHA for the installation of readline, but for the time being I am just leaving it as ??? for step three...

1. Checkout 1c2fb33043 for homebrew-core
2. $ brew install readline (should install the 8.0.0 version)
3. $ brew pin readline
4. Checkout master for homebrew-core.
5. $ brew install sphinx-doc 💥

Update: Looking at the history for homebrew-core/Formula/readline.rb, it looks like at ed947f87 it was updated to 8.0.0_1 by the bot 10 days ago, and it was originally at 8.0.0 at 1c2fb33043 (4 months ago). I updated the steps with the "theoretical steps" based on those commits, though I haven't reproduced them myself.

[MikeMcQuaid]

This is intentional behaviour.

[omnilord]

This is intentional behaviour.

I don't understand how breaking dependencies for dozens (if not hundreds) of installed softwares is considered intentional behavior. Is there a known solution to this issue outside homebrew? Because trying to install one new package triggering the reinstal of almost a hundred softwares and packages every time Readline updates is absurd.

(This has been the only critical issue I have ever had with homebrew. Amazing software otherwise!)

[iMichka]

When you pin stuff, it is a little bit at your own risk.

When we bump versions or make changes, our CI runs and we deliver a working version to our users. We have hundred of thousands of users that are fine with that setup. We ship versions we know are working correctly.

As soon as you pin a package, the dependency tree is less consistent. Stuff may work, or may not work. We want to avoid that hundreds of issues are opened to fix exotic linking issues or whatnot. Remember that we are not paid to provide support, and quite understaffed. Our CI is also quite weak, and we can not test more than we actually do.

"is absurd": it may be for you, but from our perspective, this is the right thing to do, for our own sanity, and quantity off support we can provide. So let us decide what is absurd :)

[omnilord]

That's all par for the course with most open source projects and does not solve the issue.

Are people just (for scope to the original issue) uninstalling every version of ruby they have through rbenv/rvm every single time they go to run brew upgrade? Then revisiting every project they have on their system, reinstalling those versions of ruby and running bundle install? This makes updating readline a very time consuming activity.

[NickLaMuro]

First off, @omnilord I did want to thank you for speaking up as another voice on this thread. After originally being turned down, I was pretty disheartened (to say the least), so having another person voice concerns with this does remove a bit of the imposter syndrome I did have when posting this originally. While I have some counter points to both you and @iMichka , I did want to take the time to send my gratitude your way.

---

That all said, in response @iMichka :

When you pin stuff, it is a little bit at your own risk.

I agree completely, and understand this is not something that even makes sense to try and test for. However, in addition to not only having my "outside of homebrew" items break (ruby-install items), things like postgres and node installations would break as well (pretty much anything needing a REPL), even if installed via the brew formulas, and I don't think that even the internal mechanisms in brew upgrade readline would catch that. That said, I don't think trying to provide support for every permutation of software configuration is reasonable by any stretch of the imagination and is what I would even consider as a possible fix here.

A topical Example

However, going to something I am slightly more familiar with, bundler, I think a dependency of:

gem "pg"

In a Gemfile would not mean "I only support the newest version, or BOOM!" without huge uproar from the community. In fact, that is about the most loose requirement you could imagine, and leads in to your other points in your response above of why it is bad to do that in practice. That said, I would think of "pinning" something more akin to:

gem "pg", "~> 1.0"

Or maybe something more strict that you would find in the Gemfile.lock. That said, those two example, if treated as individual dependencies would be resolvable by bundler/rubygems in just above any situation.

Unfortunately that isn't the case, and the Homebrew formula equivalents (in this theoretical example) basically assume the "go BOOM!" scenario I described above if any pinning is present, and most formula are written in a way to have zero expectations with when dealing with the version (probably based on rational provided above).

While I wouldn't use the language "absurd" based on your rational given above, I think the expectation of "always use the most recent package" just doesn't apply for most people working on legacy systems. Without any hard facts or going into much more detail, I don't think it is "absurd" to say is there are people who use brew who are working on legacy systems with packages (in production) that are out of date, and that population is not an insignificant segment of your user base.

In reference my originally issue with ruby-install that caused this issue, there is unfortunately no:

$ brew install --missing package1 package2 readline package3

Equivalent that is available in many other package managers. So brew pin was about the only workaround available to me (without running a forked version of ruby-install), but I would be willing to guess it is not unique to other language's version managers as well.

Possible Solutions:

I think a solution for the particular issue in question that I opened this up for would be simply allow installations with dependencies that are pinned. Currently "it is a little bit at your own risk" is not even possible, as brew's "dependency resolution prevents pinning from even being allowed without having to constantly enable/disable it to install dependent packages.

That said, regarding the very real concern you brought up:

As soon as you pin a package, the dependency tree is less consistent. Stuff may work, or may not work. We want to avoid that hundreds of issues are opened to fix exotic linking issues or whatnot...

... but from our perspective, this is the right thing to do, for our own sanity, and quantity off [sic] support we can provide.

is completely valid and understandable. That said, I don't think it is justification enough to completely lock out users from using a feature. Off the top of my head, this is non-exhaustive list of ideas to help prevent the issues you have described, order by "easiest to implement" to "hardest":

1. Provide a bullet item in the GitHub issue template mentioning "we do not support issues when dealing brew install issues where pinned packages are involved, and will close anything immediately that does", or in some other documentation that makes sense.
2. Provide a warning prompt when attempting to install a package that does have a pinned dependency, or possibly as part of the caveats section at the end of an install (probably make this "opt-outable".
3. Keep track of errors when executing brew install/upgrade when a pinned package is involved, and provide that in text generated when running other commands (such as brew doctor).

I would be more than willing assist in any of the above (as this is an issue that directly affects me), but with the current stance of "This is intentional behaviour." being broadcast from maintainers, it is hard for me to justify implementing even a POC for any of the above. If that happens to change, feel free to reach out here and I would be happy to take a look at providing a couple of proposed solutions.

-Nick

[MikeMcQuaid]

Are people just (for scope to the original issue) uninstalling every version of ruby they have through rbenv/rvm every single time they go to run brew upgrade? Then revisiting every project they have on their system, reinstalling those versions of ruby and running bundle install? This makes updating readline a very time consuming activity.

@omnilord You can configure ruby-build and friends to use their own readline (or none at all if you don't need it for e.g. irb) so they better handle Homebrew's readline being upgraded. Alternatively, they could use a path to Homebrew's readline that doesn't change. I'm a Ruby developer (using rbenv and ruby-build) and do not need to reinstall all my rubies on a Homebrew readline upgrade. This is not a Homebrew problem.

Because trying to install one new package triggering the reinstal of almost a hundred softwares and packages every time Readline updates is absurd.

Please adjust your tone in future, thanks.

First off, @omnilord I did want to thank you for speaking up as another voice on this thread. After originally being turned down, I was pretty disheartened (to say the least), so having another person voice concerns with this does remove a bit of the imposter syndrome I did have when posting this originally.

@NickLaMuro My apologies for any resulting imposter syndrome, it was genuinely not my intention for my actions to result in that. Triaging issues on a project as active as Homebrew on top of a job and child means my answers are often not as helpful as I'd like them to be if I had more time.

In a Gemfile would not mean "I only support the newest version, or BOOM!" without huge uproar from the community.

Homebrew is a deliberately very different system to RubyGems and the two don't really bear comparison. Homebrew has always worked this way and we do provide ways of sitting on older versions and maintaining whatever versions you want in whatever configuration you want (e.g. see brew extract).

While I wouldn't use the language "absurd" based on your rational given above, I think the expectation of "always use the most recent package" just doesn't apply for most people working on legacy systems. Without any hard facts or going into much more detail, I don't think it is "absurd" to say is there are people who use brew who are working on legacy systems with packages (in production) that are out of date, and that population is not an insignificant segment of your user base.

Homebrew's system isn't "you must always use the most recent package" (to be a bit pedantic) because it doesn't silently brew upgrade without user intervention. You decide when you want to brew upgrade (and, if you opt-out, you can decide when to brew update). You have a certain amount of control over what packages are at what version but that doesn't mean any package can work with any version of any other package.

I think a solution for the particular issue in question that I opened this up for would be simply allow installations with dependencies that are pinned.

We are never going to do this because we know in many (maybe even most) cases it will install packages that are broken which will result in torrents of issues.

1. Provide a bullet item in the GitHub issue template mentioning "we do not support issues when dealing brew install issues where pinned packages are involved, and will close anything immediately that does", or in some other documentation that makes sense.

This still results in us getting all those issues and having to manually triage them with a small team of volunteers. Additionally, it's not always obvious a pinned package is involved until later in debugging.

2. Provide a warning prompt when attempting to install a package that does have a pinned dependency, or possibly as part of the caveats section at the end of an install (probably make this "opt-outable".

Warnings that don't block user behaviour mostly end up unread.

3. Keep track of errors when executing brew install/upgrade when a pinned package is involved, and provide that in text generated when running other commands (such as brew doctor).

This is logistically very difficult.

---

I think at the root here is a misunderstanding of why Homebrew does what it does. When you brew pin bar and brew install foo without the latest version of bar you will install a binary package (bottle) that is linked against the older version of bar and will be immediately non-functional. This is why we don't allow the behaviour: it's known to be broken.

The workaround close to what's suggested/desired here would be that we could output that you could brew install --build-from-source foo and we allow building foo from source (which is dramatically more time consuming and error-prone for users and produces a bunch more issues for maintainers). On top of that, we now need to keep track that foo was built from source against an old bar and ensure that anything that depends on foo is also built from source.

If you're going to go down the route where whole parts of the dependency tree need to be built from source then you are better to follow the guidelines in https://docs.brew.sh/Versions and brew extract the versions you desire into your own tap. This is the closest equivalent to Gemfile.lock that you seem to want but comes along with the annoyance of running your own tap and building from source.

I've hopefully provided more explanation here to understand why things are the way they are and some possible workarounds you can use today. I don't expect this will satisfy your needs perfectly but I'm afraid you're asking for us to do something that seems simple but makes this package manager much harder to run so we're unwilling to do so.

[omnilord]

@NickLaMuro, oh man, good to know I'm not alone in coping with imposter syndrome!

First, let me apologize for having a panic attack yesterday and sharing my frustration here. It's just usually Homebrew is the cure for--not the cause of--package problems, and trying to solve this issue with readline has been a source of major anxiety for me in the last couple of iterations because it has become an immediate and blocking impediment that I am unable to find a solution to on my own. Homebrew has been a stable problem-free solution to my package needs for almost 6 years. This is the first and only time I've ever had a problem.

I'm a Ruby developer (using rbenv and ruby-build) and do not need to reinstall all my rubies on a Homebrew readline upgrade. This is not a Homebrew problem.

As Nick said, there is some serious imposter syndrome on this one, especially now that you make it sound like the solution is utterly simple and that makes me feel like an incompetent fool. The problem may not be a resolution in Homebrew, but the obvious (even if incorrect) solution to breaks in software that were built against packages from Homebrew was to pin readline, which is a Homebrew action.

The misconception is real, in that my interpretation of how pin functioned was wrong. My incorrect read was that pin prevents brew cleanup (which runs automatically with upgrade if it hasn't been done in the last 30 days) from deleting older versions that might be linked against elsewhere that are not tracked by homebrew. You have made it very clear that it does a lot more than just preserve a version on disk.

For posterity and others who will run into this problem, where can we find the manual to read for configuring rbenv/ruby-build to not blow up when homebrew inevitably blows away an old version of readline if it's not pinned?

[MikeMcQuaid]

First, let me apologize for having a panic attack yesterday and sharing my frustration here

I appreciate the apology, thanks.

For posterity and others who will run into this problem, where can we find the manual to read for configuring rbenv/ruby-build to not blow up when homebrew inevitably blows away an old version of readline if it's not pinned?

I'm not sure as I'm not a rbenv/ruby-build expert I'm afraid. I plan on having a rbenv plugin at some point in the future that will use Homebrew's (binary) Rubies as it's been working pretty well for me locally with the version I've hacked together.