Skip to content

Commit 74601f4

Browse files
Append paths in global config to progdb in configureCompiler v2 (#10826)
* Append paths in global config to progdb in `configureCompiler` * Add test to check global extra-prog-path is used in `configureCompiler` We add a new PackageTest for cabal-install, which makes sure that the extra-prog-path in the global config is used when resolving pkg-config instead of anything already on PATH. We do this by specifying a "bad" pkg-config to extra-prog-path that returns exit code 1. On Windows, Cabal logs are identical in the following two cases: - the override didn't work, and pkg-config was not found on the system - the override did work, and querying it failed (expectedly) To work around this, we add a "good" pkg-config script to PATH before- hand, so that cabal succeeds in the first case and the logs differ. This test is a stripped down reproduction of what happens when a user on Windows adds MSYS2 paths to extra-prog-path but not the system PATH. This is a reasonable setup (often, adding MSYS2 to system PATH is not good practice), and is even created automatically by GHCup. * Refactor `configureCompiler` path addition for cleaner variable names * Remove unnecessary build target from ExtraProgPathGlobal test command * Fix ExtraProgPathGlobal test failing on Mac/Unix for missing +x perms * Rename pr-10790 changelog to pr-10826 as it superceded * Add hashtag before PR/issue numbers in changelog for 10826 --------- Co-authored-by: Javier Sagredo <[email protected]>
1 parent c0d52b2 commit 74601f4

File tree

13 files changed

+174
-7
lines changed

13 files changed

+174
-7
lines changed

cabal-install/src/Distribution/Client/ProjectPlanning.hs

+11-7
Original file line numberDiff line numberDiff line change
@@ -476,6 +476,7 @@ configureCompiler
476476
{ projectConfigHcFlavor
477477
, projectConfigHcPath
478478
, projectConfigHcPkg
479+
, projectConfigProgPathExtra
479480
}
480481
, projectConfigLocalPackages =
481482
PackageConfig
@@ -499,25 +500,28 @@ configureCompiler
499500
)
500501
$ do
501502
liftIO $ info verbosity "Compiler settings changed, reconfiguring..."
502-
let extraPath = fromNubList packageConfigProgramPathExtra
503-
progdb <- liftIO $ prependProgramSearchPath verbosity extraPath [] defaultProgramDb
504-
let progdb' = userSpecifyPaths (Map.toList (getMapLast packageConfigProgramPaths)) progdb
505-
result@(_, _, progdb'') <-
503+
progdb <-
504+
liftIO $
505+
-- Add paths in the global config
506+
prependProgramSearchPath verbosity (fromNubList projectConfigProgPathExtra) [] defaultProgramDb
507+
-- Add paths in the local config
508+
>>= prependProgramSearchPath verbosity (fromNubList packageConfigProgramPathExtra) []
509+
>>= pure . userSpecifyPaths (Map.toList (getMapLast packageConfigProgramPaths))
510+
result@(_, _, progdb') <-
506511
liftIO $
507512
Cabal.configCompilerEx
508513
hcFlavor
509514
hcPath
510515
hcPkg
511-
progdb'
516+
progdb
512517
verbosity
513-
514518
-- Note that we added the user-supplied program locations and args
515519
-- for /all/ programs, not just those for the compiler prog and
516520
-- compiler-related utils. In principle we don't know which programs
517521
-- the compiler will configure (and it does vary between compilers).
518522
-- We do know however that the compiler will only configure the
519523
-- programs it cares about, and those are the ones we monitor here.
520-
monitorFiles (programsMonitorFiles progdb'')
524+
monitorFiles (programsMonitorFiles progdb')
521525

522526
-- Note: There is currently a bug here: we are dropping unconfigured
523527
-- programs from the 'ProgramDb' when we re-use the cache created by
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
module MyLibrary () where
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
packages: *.cabal
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: CheckExtraProgPathGlobal
2+
version: 0.1
3+
license: BSD3
4+
license-file: LICENSE
5+
author: Yuto Takano
6+
maintainer: Yuto Takano
7+
synopsis: Check Extra Prog Path Global
8+
category: PackageTests
9+
build-type: Simple
10+
cabal-version: 2.0
11+
12+
description:
13+
Check that Cabal recognizes an override of the prog path.
14+
15+
Library
16+
pkgconfig-depends: zlib
17+
default-language: Haskell2010
18+
build-depends: base <5.0
19+
exposed-modules:
20+
MyLibrary
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/bin/sh
2+
3+
set -eu
4+
5+
# ugly, but "good enough" for this test
6+
# This will need to be updated whenever cabal invokes pkg-config
7+
# in new ways
8+
case "$*" in
9+
'--version')
10+
echo 2.1.0 # whatever
11+
;;
12+
13+
'--variable pc_path pkg-config')
14+
echo '.'
15+
;;
16+
17+
'--list-all')
18+
printf 'zlib zlib - zlib compression library\n'
19+
# \256 = \xAE is the iso-8859-1 (latin-1) encoded version of U+00AE,
20+
# i.e. the "registered sign": ®
21+
# This resulted in problems, see #9608
22+
printf 'vpl Intel\256 Video Processing Library - Accelerated video decode, encode, and frame processing capabilities on Intel\256 GPUs\n'
23+
# \360 = \xF0 is latin-1 for ð; this is orð, Icelandic for "word"/"words".
24+
printf 'or\360 Icelandic characters\n'
25+
;;
26+
27+
'--modversion '*)
28+
shift # drop the --modversion
29+
for arg; do
30+
case "$arg" in
31+
zlib) echo 1.3; ;; # whatever
32+
vpl) echo 2.10; ;; # whatever
33+
# No entry for orð here; let's not even try to match on that
34+
*)
35+
echo >&2 "Package $arg was not found in the pkg-config search path."
36+
exit 1
37+
esac
38+
done
39+
;;
40+
41+
# Ignore some stuff we're not implementing
42+
'--cflags '*) ;;
43+
'--libs '*) ;;
44+
45+
*)
46+
echo >&2 "pkg-config: unrecognised arguments $* (this is an incomplete shim)"
47+
exit 1
48+
;;
49+
esac
Binary file not shown.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
path = "FINDSH/sh.exe"
2+
args = "SCRIPTSWINPATHDIR/pkg-config"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
3+
exit 1;
Binary file not shown.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
path = "FINDSH/sh.exe"
2+
args = "SCRIPTSDIR/pkg-config"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# cabal v2-build
2+
Configuration is affected by the following files:
3+
- cabal.project
4+
Warning: cannot determine version of <ROOT>/scripts/pkg-config :
5+
""
6+
Warning: cannot determine version of <ROOT>/scripts/pkg-config :
7+
""
8+
Resolving dependencies...
9+
Error: [Cabal-7107]
10+
Could not resolve dependencies:
11+
[__0] next goal: CheckExtraProgPathGlobal (user goal)
12+
[__0] rejecting: CheckExtraProgPathGlobal-0.1 (pkg-config package zlib-any is needed but no pkg-config executable was found or querying it failed)
13+
[__0] fail (backjumping, conflict set: CheckExtraProgPathGlobal)
14+
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: CheckExtraProgPathGlobal (2)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
import Test.Cabal.Prelude
2+
import System.Directory
3+
4+
-- Test that extra-prog-path in the global cabal config (~/.cabal/config)
5+
-- overrides the path for pkg-config while resolving a dependency on zlib.
6+
-- Specifically, this test emulates the scenario on Windows where pkg-config
7+
-- doesn't exist natively and must be provided via extra-prog-path.
8+
main = cabalTest $ do
9+
env <- getTestEnv
10+
let
11+
testDir = testCurrentDir env
12+
tmpDir = testTmpDir env
13+
-- Scripts is where we put a bad pkg-config
14+
scripts = tmpDir </> "scripts"
15+
-- Scripts-Winpath contains a mock pkg-config that returns good results.
16+
-- We add this to PATH on Windows before executing the test so we can tell
17+
-- apart when Cabal uses the override or not.
18+
scripts_winpath = tmpDir </> "scripts-winpath"
19+
20+
-------------------------
21+
-- Workaround for the fact that, on Windows, Cabal will only look for
22+
-- .exe files to satisfy executable dependencies. So we have to create a
23+
-- shim pkg-config.exe file in 'script'. This is a thin wrapper (which is
24+
-- explicitly added to Git) that calls whatever is defined in the .shim
25+
-- file. In our case, we rewrite the .shim file as below so that the
26+
-- pkg-config script is executed using sh.
27+
when isWindows $ do
28+
mb_sh <- fmap takeDirectory <$> liftIO (findExecutable "sh")
29+
case mb_sh of
30+
Nothing -> skip "no sh"
31+
Just sh -> do
32+
let escape = concatMap (\c -> case c of '\\' -> "\\\\\\\\"; x -> [x])
33+
void $ shell "sed" [ "-i", "-e", "s/FINDSH/" <> escape sh <> "/g", escape (scripts </> "pkg-config.shim"), escape (scripts_winpath </> "pkg-config.shim") ]
34+
void $ shell "sed" [ "-i", "-e", "s/SCRIPTSDIR/" <> escape scripts <> "/g", escape (scripts </> "pkg-config.shim") ]
35+
void $ shell "sed" [ "-i", "-e", "s/SCRIPTSWINPATHDIR/" <> escape scripts_winpath <> "/g", escape (scripts_winpath </> "pkg-config.shim") ]
36+
37+
-- End of Windows workaround
38+
------------------------------
39+
40+
-- On Windows, we want it to find the "scripts/pkg-config" (which will
41+
-- return exit code 1), instead of what we add to the path
42+
-- ("scripts-winpath/pkg-config", which will return exit code 0). This is
43+
-- because Windows doesn't have a system pkg-config by default. If we didn't
44+
-- add a known-to-be-good mock pkg-config in the path, we wouldn't be able
45+
-- to tell apart from Cabal logs whether it wasn't able to find pkg-config
46+
-- at all (test fail) or whether the override worked and it found the bad
47+
-- one but couldn't query it (test success).
48+
--
49+
-- On other systems, we want it to find "scripts/pkg-config" (exit code 1)
50+
-- instead of the system pkg-config (success).
51+
let wrap_test = if isWindows then addToPath scripts_winpath else id
52+
53+
-- Add global config override in ~/.cabal/config (in the test environment).
54+
liftIO $ appendFile (testUserCabalConfigFile env) $
55+
"\nextra-prog-path: " ++ scripts
56+
liftIO $ putStrLn $ testUserCabalConfigFile env
57+
58+
-- On correct behaviour, cabal should fail because it found our exit-code-1
59+
-- pkg-config through the global extra-prog-path.
60+
fails $ wrap_test $ cabal "v2-build" []

changelog.d/pr-10826

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
synopsis: Prepend the global extra paths to the `ProgramDb` sooner
2+
packages: cabal-install
3+
prs: #10826
4+
issues: #9800
5+
significance: significant
6+
7+
description: {
8+
9+
- The initial configuration of programs was done without the `extra-prog-path` from the global cabal configuration. This meant that in some cases, some executables were not found. In particular this manifested as Windows users who didn't add the MSYS2 paths to the global PATH couldn't make use of `pkg-config`.
10+
11+
}

0 commit comments

Comments
 (0)