allow-newer causes setup failure

[bergmark]

It seems like allow-newer doesn't always allow-newer! The essential parts of the reproduction below are:

• My package depends on mmorph
• transformers-0.5.0.0 is in extra-deps
• mmorph Doesn't allow transformers-0.5.0.0
• I've enabled allow-newer: True
• Running stack build errors out with
  Configuring mmorph-1.0.4... setup-Simple-Cabal-1.22.5.0-ghc-7.10.3: At least the following dependencies are missing: transformers >=0.2.0.0 && <0.5 && ==0.5.0.0

Currently the only workaround I've found is to add mmorph as a local package and relaxing the transformers constraint in its cabal file.

Full reproduction:

V~/repos> ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.10.3

V~/repos> stack --version
Version 1.0.0 x86_64

V~/repos> git clone git@github.com:bergmark/stack-allow-newer.git
Cloning into 'stack-allow-newer'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 1), reused 10 (delta 1), pack-reused 0
Receiving objects: 100% (10/10), done.
Resolving deltas: 100% (1/1), done.
Checking connectivity... done.

V~/repos> cd stack-allow-newer/

V~/r/stack-allow-newer master> ls
LICENSE    Setup.hs   src        stack.yaml test.cabal

V~/r/stack-allow-newer master> stack build --dry-run
WARNING: Ignoring out of range dependency (allow-newer enabled): transformers-0.5.0.0. mmorph requires: >=0.2.0.0 && <0.5
No packages would be unregistered.

Would build:
mmorph-1.0.4: database=local, source=package index, after: transformers-0.5.0.0
test-0.1.0.0: database=local, source=/Users/adam/repos/stack-allow-newer/, after: mmorph-1.0.4
transformers-0.5.0.0: database=local, source=package index

No executables to be installed.

V~/r/stack-allow-newer master> stack build -v
Version 1.0.0 x86_64
2015-12-30 17:02:07.866168: [debug] Checking for project config at: /Users/adam/repos/stack-allow-newer/stack.yaml @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Config src/Stack/Config.hs:577:9)
2015-12-30 17:02:07.867348: [debug] Loading project config file stack.yaml @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Config src/Stack/Config.hs:600:13)
2015-12-30 17:02:07.868859: [debug] Trying to decode /Users/adam/.stack/build-plan-cache/x86_64-osx/nightly-2015-12-30.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2015-12-30 17:02:07.890366: [debug] Success decoding /Users/adam/.stack/build-plan-cache/x86_64-osx/nightly-2015-12-30.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2015-12-30 17:02:07.890533: [debug] Trying to decode /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2015-12-30 17:02:08.246045: [debug] Success decoding /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2015-12-30 17:02:08.261555: [debug] Run process: ghc --info @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.341266: [debug] Run process: ghc --numeric-version @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.386835: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.415385: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.449187: [debug] Run process: locale -a @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.464777: [debug] Checking resolver: nightly-2015-12-30 @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Source src/Stack/Build/Source.hs:164:17)
2015-12-30 17:02:08.465002: [debug] Trying to decode /Users/adam/.stack/build-plan-cache/x86_64-osx/nightly-2015-12-30.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2015-12-30 17:02:08.580945: [debug] Success decoding /Users/adam/.stack/build-plan-cache/x86_64-osx/nightly-2015-12-30.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2015-12-30 17:02:08.585757: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.639184: [debug] Ignoring package haskeline due to wrong location: (Nothing,Local) @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2015-12-30 17:02:08.639337: [debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1 @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2015-12-30 17:02:08.639500: [debug] Ignoring package transformers due to wrong location: (Nothing,Local) @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2015-12-30 17:02:08.639579: [debug] Ignoring package Cabal due to wanting version 1.22.6.0 instead of 1.22.5.0 @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2015-12-30 17:02:08.675123: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /Users/adam/.stack/snapshots/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb dump --expand-pkgroot @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.713516: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb dump --expand-pkgroot @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:08.742686: [debug] Trying to decode /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2015-12-30 17:02:09.072138: [debug] Success decoding /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2015-12-30 17:02:09.191892: [warn] WARNING: Ignoring out of range dependency (allow-newer enabled): transformers-0.5.0.0. mmorph requires: >=0.2.0.0 && <0.5 @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.ConstructPlan src/Stack/Build/ConstructPlan.hs:148:11)
2015-12-30 17:02:09.192822: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_AiuFEtMyGSB1lMos3PapcF:System.Process.Read src/System/Process/Read.hs:267:3)
2015-12-30 17:02:09.220260: [debug] Trying to decode /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2015-12-30 17:02:09.567253: [debug] Success decoding /Users/adam/.stack/indices/Hackage/00-index.cache @(stack_AiuFEtMyGSB1lMos3PapcF:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)
2015-12-30 17:02:09.587612: [info] mmorph-1.0.4: configure @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Execute src/Stack/Build/Execute.hs:668:23)
2015-12-30 17:02:09.588108: [debug] Run process: /Users/adam/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 configure --with-ghc=/Users/adam/.ghc-7.10.3/bin/ghc --with-ghc-pkg=/Users/adam/.ghc-7.10.3/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/Users/adam/.stack/snapshots/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb --package-db=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb --libdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/lib --bindir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/bin --datadir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/share --libexecdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/libexec --sysconfdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/etc --docdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --htmldir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --haddockdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --dependency=base=base-4.8.2.0-cc43d8e06aa74e9c4c0132becc49ee25 --dependency=transformers=transformers-0.5.0.0-801823ea63f393cac32b591c99f746d0 @(stack_AiuFEtMyGSB1lMos3PapcF:Stack.Build.Execute src/Stack/Build/Execute.hs:812:21)
Progress: 1/2
--  While building package mmorph-1.0.4 using:
      /Users/adam/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 configure --with-ghc=/Users/adam/.ghc-7.10.3/bin/ghc --with-ghc-pkg=/Users/adam/.ghc-7.10.3/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/Users/adam/.stack/snapshots/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb --package-db=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/pkgdb --libdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/lib --bindir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/bin --datadir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/share --libexecdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/libexec --sysconfdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/etc --docdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --htmldir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --haddockdir=/Users/adam/repos/stack-allow-newer/.stack-work/install/x86_64-osx/nightly-2015-12-30/7.10.3/doc/mmorph-1.0.4 --dependency=base=base-4.8.2.0-cc43d8e06aa74e9c4c0132becc49ee25 --dependency=transformers=transformers-0.5.0.0-801823ea63f393cac32b591c99f746d0
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/adam/repos/stack-allow-newer/.stack-work/logs/mmorph-1.0.4.log

    Configuring mmorph-1.0.4...
    setup-Simple-Cabal-1.22.5.0-ghc-7.10.3: At least the following dependencies
    are missing:
    transformers >=0.2.0.0 && <0.5 && ==0.5.0.0

[mgsloan]

See haskell/cabal#2023 for some background on this. Looks like we need to pass in --exact-configuration when doing --allow-newer and on Cabal >= 1.19.2 (in other words, when using >= lts-3).

I think we may actually want to always use --exact-configuration, since we're always giving an explicit list of dependencies. This way, we'll also be more likely to catch bugs with this, than if it only happens when --allow-newer is specified.

Searching cabal's source code revealed this - looks like --exact-configuration also has the affect of requiring that all the cabal flags be specified. As such, we'd need to be sure to specify all the flags, which I don't think currently happens (maybe it does, though!).

[mgsloan]

Just encountered this myself, definitely ought to get fixed..

[acfoltzer]

Hit this one too; would be very useful with the upcoming GHC release

[radix]

@mgsloan said

See haskell/cabal#2023 for some background on this. Looks like we need to pass in --exact-configuration when doing --allow-newer and on Cabal >= 1.19.2 (in other words, when using >= lts-3).

I ran into this problem on lts-2.22 while trying to depend on snap-0.13.2.7 with an overridden version of mtl-2.2.1. I'm not really sure I understand your qualification here but I just want to add the datapoint that this happens on LTS <3 as well.

[mgsloan]

@radix It won't work for LTS < 3, and I don't think there's anything we can do about it. This also affected cabal-install - see haskell/cabal#2023

--allow-newer suddenly requires Cabal >=1.19.2 to be available

Setup.hs scripts compiled against Cabal < 1.19.2 don't support --exact-configuration which is needed for --allow-newer to work.

[radix]

@mgsloan ah, I think I misparsed your original comment.

[tolysz]

This is really bad. It is very difficult to build anything with ghc-8 without this option :(

[mgsloan]

@tolysz Thanks for bumping this issue. I'd worked on it, but ran into some issues and hadn't got back to it. I think it's fixed now. Currently waiting on a ghc 8.0 build of stack :)

[mgsloan]

Pushed a fix to the fix.