Linking error when dependency uses own 'extra-library'

[javierdiegof]

General summary/comments (optional)

I get a linking error by GNU gold when trying to add an external library:

/usr/bin/x86_64-linux-gnu-ld.gold: error: cannot find -lCacBDD
collect2: error: ld returned 1 exit status

I tried it in my regular Ubuntu 18.04 machine and when that failed i used a clean virtual machine installation of the same distribution, I get the same error on both.

Steps to reproduce

With stack installed, I created a new project with a template:

$ stack new my_project_name new-template

Then import the module I want in src/Lib.hs (which is HasCacBDD):

module Lib
    ( someFunc
    ) where

import Data.HasCacBDD
someFunc :: IO ()
someFunc = putStrLn "someFunc"

Then I include the dependency in the package.yaml file:

dependencies:
- base >= 4.7 && < 5
- HasCacBDD

And since the package is not included by default in any release of stack, I add the extra dependency in
the stack.yaml file:

extra-deps:
- HasCacBDD-0.1.0.0

When I try to build the project, it simply does not work.

Expected

From a beginners point of view, this configuration looks very similar to the acme-missiles example
in the docs, nevertheless, that example works perfectly and this one does not.

I would expect it to build ok or at least show me some infomative message that would help me solve the problem.

Actual

A linker error.

$ stack build
HasCacBDD-0.1.0.0: configureHasCacBDD-0.1.0.0: buildHasCacBDD-0.1.0.0: copy/register
Building all executables for `haskell-smv' once. After a successful build of all of them, only specified executables will be rebuilt.
haskell-smv-0.1.0.0: configure (lib + exe)
Configuring haskell-smv-0.1.0.0...
haskell-smv-0.1.0.0: build (lib + exe)Preprocessing library for haskell-smv-0.1.0.0..Building library for haskell-smv-0.1.0.0..
[1 of 2] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Lib.o )
[2 of 2] Compiling Paths_haskell_smv ( .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/autogen/Paths_haskell_smv.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Paths_haskell_smv.o )
/usr/bin/x86_64-linux-gnu-ld.gold: error: cannot find -lCacBDD
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)
Completed 2 action(s).

--  While building custom Setup.hs for package haskell-smv-0.1.0.0 using:
      /home/javier/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.2.0.1 build lib:haskell-smv exe:haskell-smv-exe --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

As I mentioned before, the acme-missiles example works fine, and also, I am also able to build the package i'm trying to add directly (as explained in the docs).

Stack version

I tried with both the latests LTS:

$ stack --version
Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2

and also with the latest nightly --resolver nightly-2018-07-09 getting the same error on both.

Method of installation

• Official binary, downloaded from stackage.org or fpcomplete's package repository

Additional (maybe useful)

HasCacBDD is a binding to a C package using the FFI. Looking
at the error, it looks like the missing file is a library from that C package. Still, the error seems odd because if you execute the conventional stack unpack HasCacBDD-0.1.0.0 and build it, all the code works just fine. Maybe the issue could be further specified by the question:

Why is it the case that adding the external package as a dependency does not work the same way as when stack unpack is used?

Note: Looking at the code, the Setup.hs of the original package seems to be building and copying the mentioned libCacBDD.a static library into the correct folder inside the .stack-work folder (it does so by running some Makefile's on the original C code and then moving the built library into the folder).

When the package is included as an extra-dep, this libCacBDD.a library is indeed being stored at .stack-work/install/x86_64-linux/resolver_version/ghc_version/lib/x86_64-linux-ghc_version/included_package_name/library.a of my project, so probably the only thing that I need to do is to add this folder to library dependencies. It seems awkard and error prone to write the path I mentioned above, since it dependes on the particular resolver, operating system, and other settings of my project. Is there a more robust way to add this folder to my dependencies, maybe in an automatic way?
Thanks in advance!

[dbaynard]

I can reproduce (with a different resolver, on ghc 8.2.2).

I am, though, able to build HasCacBDD with just a stack build HasCacBDD.

I'll find somebody who knows more — it should work automatically.

[dbaynard]

It also doesn't work with a custom snapshot.

[javierdiegof]

Thank you very much for your response.

The command stack build HasCacBDD is indeed building the package directory inside the .stack-work folder and pasting the static library inside it. But, if any of the exposed functions of the package is used when interpreting the src\Lib.hs file, the missing static library is again a problem (you can try with the constant function bot):

*Lib> bot
can't load .so/.DLL for: libCacBDD.so (libCacBDD.so: cannot open shared object file: No such file or directory)

For a reason I'm not aware of, it asks for a shared library instead of the static one i would expect it to.

Thank's again.

[dbaynard]

Have you tried playing with ghc-options? I've directed the linker in the past, which you could try by adding ghc-options: -optl-Wl,-rpath,$ORIGIN/.stack-work to your cabal file the following to your package.yaml?

ghc-options:
- -optl-Wl,-rpath,$ORIGIN/.stack-work

It would be helpful to have the same example project to work with, so would you please make a repo on github that we can use as a minimum example?

[javierdiegof]

Hi. I'm sorry for answering so late. I was able to make it work, but the solution was sub-optimal.

Basically, what I did was to change the information of the package_name.cabal (equivalently package.yaml ) of the original package to the information of my own project. The original project calls the Setup.hs that puts the library in the correct folder, so by changing the cabal file, the Setup.hs now uses the information of my project (which it extracts from the .cabal file) and puts the library in the correct place.

Of course, that's not a very good solution:

1. I am not able to just call the package to use it.
2. I had to tinker with the original .cabal file in a very ad-hoc way.
3. If a new version of the package I'm using comes out, i will have to rewrite the whole thing again.

I don't know how to solve in a better way. On the whole, I think a tutorial on the different elements of a Setup.hs file (that goes in a little bit more detail than the Hackage Docs) would have been very useful, since I think is the only way to put the library in the correct place when using the stack build command.

I'm willing to contribute in helping in any way I can to make it easier for other people that encounter the same trouble (C binding packages will probably be the ones encounter it the most).

[m4lvin]

For a working example of using HasCacBDD in another stack project, see https://github.com/jrclogic/SMCDEL/. (I am the author of HasCacBDD and SMCDEL 😉)

Still, a bug in stack might be that extra-deps and a location with extra-dep: true behave differently. That is, if I change https://github.com/jrclogic/SMCDEL/blob/a5aaee888670fa34e248b203b89c3c5ed97611f3/stack.yaml#L9 as follows ...

-- location:
-    git: https://github.com/m4lvin/HasCacBDD.git
-    commit: 23405da09876356569fbbb4f150e33199e5546f9
-  extra-dep: true
+
+extra-deps:
+- HasCacBDD-0.1.0.0

... it no longer builds but fails with:

/usr/bin/ld.gold: error: cannot find -lCacBDD
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)

(Note that the git commit and the version 0.1.0.0 on hackage have identical code besides the README.)

Maybe someone can explain the difference between the two extra-dep(s) methods?

PS: Some information how the Setup.hs trick works can be found at https://codinginfinity.me/post/2015-04-18/haskell_and_cpp and the accompanying repository https://github.com/jakubfijalkowski/haskell-and-cpp

[javierdiegof]

Just to add to what Malvin has commented, the error happens even when using the extra-deps now recommended syntax for packages not in Stackage, that is, the following changes:

-extra-deps:
-  - HasCacBDD-0.1.0.0
+
+extra-deps:
+  - git: https://github.com/m4lvin/HasCacBDD.git
+    commit: 23405da09876356569fbbb4f150e33199e5546f9

Make the package work correctly. So probably the question to ask is "Why is it working correctly with a
Github repo and not with a Hackage one?"

[dbaynard]

Could it be a problem with the package itself?

[javierdiegof]

Possibly, we should check with Malvin.

…

On Thu, Sep 20, 2018 at 1:34 AM dbaynard ***@***.***> wrote: Could it be a problem with the package itself? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4141 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQLbpg48VQuALqvwaKtBlEaSvbtbR9Ztks5uczb4gaJpZM4VIaLu> .

[m4lvin]

I tried to make a more minimal example , but still using HasCacBDD, see here:
https://gist.github.com/m4lvin/54fe773cf7c813354c0889a5bf2bc2e7

Note that:

1, There is different behavior even between only giving the package name and version to get it from Hackage vs. giving the full URL to the same Hackage package?!
2. In the minimal example the difference is only about whether stack ghci works. But the project can be built, no matter which method is used to pull in HasCacBDD. This seems to be because the importing project does not use template Haskell (like SMCDEL does) and also does not have a custom Setup.hs (as SMV-Haskell had, I think).

From https://docs.haskellstack.org/en/stable/yaml_configuration/#packages-and-extra-deps I understood that all methods to get an extra-deps package should do the same, but apparently they don't. Maybe you can point us to the relevant stack source code to see what happens?

To test whether this also happens with other packages we need another package that (i) builds a shared library via Setup.hs (ii) is available on Hackage and (iii) is also available in a git repository. I don't know of any immediately. For cudd 0.1.0.2 (binding to CUDD 2.5.0) by @adamwalker I have done the same at https://github.com/m4lvin/cudd but this is not on Hackage of course, as it is not my package.

Edit: Maybe https://github.com/jakubfijalkowski/hlibsass would be another example?
Edit2: Not really, it is already included in stackage, so the extra-deps has no effect.

[m4lvin]

Are there any news about this or hints how to debug it further? Can someone point us to the relevant source code of stack?

To summarize the question: Why does stack treat extra-deps given as a (git or https) url differently than those given as name-version referring to Hackage?

[dbaynard]

Why does stack treat extra-deps given as a (git or https) url differently than those given as name-version referring to Hackage?

With hackage, all the information is present in the hackage tarball. With repository extra-deps, stack has to download the repo and then process it. This is changing, though.

No news. That said, there have been new releases since then, and it may also be worth trying on the master branch as that completely changes the way git dependencies work.

[m4lvin]

it may also be worth trying on the master branch as that completely changes the way git dependencies work.

Thank you for the hint! I just tried to build SMCDEL with stack from the master branch and now it fails with /usr/bin/ld.gold: error: cannot find -lCacBDD no matter how I add the dependency on HasCacBDD in stack.yaml. I guess that in stack 2 the git or http dependencies work more similar to Hackage dependencies?

Hence I will not be able to build my projects with stack 2 when it comes out 😿

I also want to clarify that I am by no means sure the problem is not with my package. If someone can look at https://github.com/m4lvin/HasCacBDD/blob/master/HasCacBDD.cabal etc. and has a better idea how to make a package including Haskell-to-C-to-C++ FFI more stack friendly, I would be very happy!

[dbaynard]

🤦 that's not good! Does HasCacBDD still build directly, just not as a dependency?

I wonder whether it has anything to do with the custom setup? Specifically, the following line jumped out at me:

https://github.com/m4lvin/HasCacBDD/blob/b5a9ab0ffc0e9cb41f18523036e476208f0a580a/Setup.hs#L51

Building as a snapshot dependency would (I believe) place the library into the ~/.stack directory — do you know exactly where it ends up?

[m4lvin]

🤦 that's not good! Does HasCacBDD still build directly, just not as a dependency?

It does still build directly. I can run stack build && stack exec ghci and then import Data.HasCacBDD and for example var 5 works fine. But what I only realise now: This uses libCacBDD.a from the c/ folder and not from libPref - at least it no longer works if I delete the one in c/.

If I build SMCDEL depending on HasCacBDD with the current stack master, this debugging line prints /tmp/stack22766/HasCacBDD-0.1.0.0/c and later I see this message:

Installing library in /home/malvin/.stack/snapshots/x86_64-linux/f564b468707d09eb8ba80953127e6f463a9f54d124aa10540390f11fd34bf7d4/8.4.4/lib/x86_64-linux-ghc-8.4.4/HasCacBDD-0.1.0.0-F6x9l4bvFegDcTnUYJDPu2

That latter folder does then indeed contain libCacBDD.a, but this file seems not to be used, because this folder is not searched for libraries when compiling SMCDEL with HasCacBDD as a dependency.

An easier example than HasCacBDD might be https://github.com/jakubfijalkowski/haskell-and-cpp from where I copied the method of the flying .a file. Here is a minimal example depending on this toy library which shows the same regression from stack 1.9.3 to master: https://github.com/m4lvin/depender ⬅️

Note that I am running stack master and stack 1.9.3 with the same ~/.stack folder. Could that cause problems?

[m4lvin]

Given that stack 2 was now released, this became a bit more urgent for me. Could someone look at https://github.com/m4lvin/depender or give me other ideas how to debug the problem?

[dbaynard]

@m4lvin I'm digging in to this. Do you think it could be related to #4855?

[m4lvin]

Thank you! I am unsure about the relation to #4855. Indeed, stack ghci also does not load the c library, but the error is different. I think stack is at least adding the correct -l option, but the library is not found because the paths are not set. This is what I get with the latest stack version and stack ghci on https://github.com/m4lvin/depender:

depender> initial-build-steps (lib)
Configuring GHCi with the following packages: depender
GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
<command line>: can't load .so/.DLL for: libext.so (libext.so: cannot open shared object file: No such file or directory)

(whereas with old stack 1.9.3 this works)

[m4lvin]

I did some more experiments. It seems stack 2 also ignores any extra-lib-dirs setting in the stack.yaml of an included package, but in stack 1.9.3 I can use that to make stack ghci work.

In general, I think stack build and stack ghci are two separate issues with dependencies that "carry" their own library, because extra-lib-dirs might be configured in different places (package.yaml/.cabal file or custom Setup.hs or stack.yaml).

I only now learned that stack ghci does not run Setup.hs -- see #970 -- but that could sometimes be worked around with stack exec ghci, though no longer with stack 2.

Sorry for all the confusion 😕 (I also stumbled into haskell/cabal#1317 but I think the problem here is whether the extra-lib-dirs is passed at all.)

---

update: some good news: https://github.com/jakubfijalkowski/hsass does build with stack 2, even though it depends on https://github.com/jakubfijalkowski/hlibsass which also uses a custom Setup.hs to make a C and C++ library. Maybe @jakubfijalkowski can help?

---

next update: spying on getting inspiration from hlibsass was a good strategy 😄
m4lvin/haskell-and-cpp@36317c4 seems to solve the problem. I guess the new stack builds extra-deps[*] in /tmp/... but then copies the binaries to ~/.stack or ./.stack-work and then paths made using getCurrentDirectory() become useless. Moreover, it seems the postConf hook needs to be disabled to preseve the extraLibDir settings. Both of these problems only happen for dependencies, because if the package with the extra library is the main package currently being built, then no /tmp folder is used, I think.

In total, this might mean there is no bug in stack after all 😉
I will try fixing HasCacBDD+SMCDEL in the same way and then report back.

[*] with one exception that makes these problems hard to debug: extra-deps given as relative or absolute paths (i.e. not git, https, hackage etc.) are built in place, not in /tmp, hence also not triggering such problems.

[m4lvin]

Indeed this solves the problem. HasCacBDD 0.1.0.2 can now be used inextra-deps with both 1.9.3 and the latest stack 🎉

[jakubfijalkowski]

I was on vacations and couldn't respond, sorry. I'm glad that it was fixed (I am amazed that my 5-years-old post is still usable). :)