Figure out project initialization

[snoyberg]

We have a number of different issues all touching on the topic of project initialization (#116 #137 #228 #229). I'd like to use this issue to take a step back and discuss goals at a higher level to try and come up with solution that addresses everything.

It seems like we have the following different use cases that are overlapping:

• Start a brand new project without any source code preexisting
• Take a preexisting Cabal package (with a single .cabal file) and create a stack.yaml file
• Take a preexisting mega-repo (multiple .cabal files) and create a stack.yaml file
• Figure out the best resolver settings to use for 1 or more existing .cabal files
• Deal well with the case where no LTS or Nightly is found that can compile the user's code
• Upgrading a stack.yaml to a newer snapshot
• Installing GHC

Possibly related ideas on the table already are:

• stack new create a complete project from a template, which addresses the "brand new project" case
• stack build automatically creates a stack.yaml already, but additionally throw in a stack init command to be more explicit about it, and let that command have more flags to control behavior. @gregwebs brought up the idea of naming this stack config instead, and letting it double as a means of modifying an existing stack.yaml
• stack setup will download and install GHC if missing. There was some discussion about that command name being confusing
• For cases where no LTS or Nightly works, we currently fall back to an LTS. Instead, we could fall back to using a dependency solver to create a set of extra-deps (or use custom snapshots when available Custom snapshots #111)

I think that summarizes the goals and proposals until now. Before diving into solving all of these things coherently, does anyone want to add more requirements to the mix?

(Assigning to myself to drive the discussion forward, but others should feel free to jump in, make decisions, and implement things.)

[chrisdone]

I'm in favour of init meaning "setup my stuff" (GHC, config), and new meaning "make a new project" (as opposed to cabal's naming of "init = make a new project").

And config for editing the configuration a la Git still seems good to me.

Brainstorming: One point to consider is whether to make some of these commands interactive or not, with flags to make them not interactive. Part of what made everyone love using Darcs was its helpful interactive by default process, whereas Git requires a flag to be interactive. Anyway, the reason I bring it up is it makes it easier to have commands choose a codepath based on user feedback instead of trying to figure out the perfect behaviour for a command name or hiding behaviours in flags that might be missed.

[snoyberg]

While generally I like the idea of stack not being interactive, specifically project initialization is a great place to change that and be as helpful/interactive as possible.

[mboes]

Yes so long as the user is not asked too many questions. I find even cabal init too pesky.

Related: my fat fingers often cd me into the wrong repository, say src/ instead src/somepackage. In this case stack build cheerfully goes ahead and downloads random data (as part of init), when I would prefer it to just say "dude, I'm not sure what you want me to do here."

[snoyberg]

Oh, that reminds me: I should have referenced #231. My proposal there was to not automatically create a stack.yaml file when there are no .cabal files found.

[andrewthad]

On the topic of the last bullet point (falling back), this is what I would prefer to see. If it doesn't work with the snapshot specified in stack.yaml, print out an indication that this has happened and then fail. Maybe you could reenable the falling back behavior with a flag (like --allow-fallback or something like that).

However, the allow-fallback solution is kind of weird because you would need to specify the flag every time when what you should really do is just fix the resolver value in stack.yaml. So I think that an even better solution than allow-fallback is figuring out a suitable resolver and displaying this recommendation to the user. Here is roughly the message I have in mind:

The package could not be built with lts-2.13. However, it has been detected that lts-2.8 will correctly 
resolve dependencies. Change the resolver in stack.yaml to fix this.

And I will throw out one more idea. It may be nice to have a command stack list-resolvers (that's a pretty bad name, but you get the idea). It would just find which package sets will cause dependencies to resolve. So the output would be something like:

Detecting suitable build plans...
  lts-2.13 yes
  lts-2.12 yes
  lts-2.11 yes
  lts-2.10 no
  lts-2.9 no
  ...

And of course it would have the nightly stuff in there as well. You'd probably want a way to filter what it tries or how many it will try.

[chreekat]

+1 to:

cmd	domain
init	set up global stack stuff (i.e. rename setup to init)
config	set up or modify stack.yaml (needs a .cabal)
new	lay out a new project (generates .cabal and sundries)
build	just builds, requiring stack.yaml, but suggests new or config when appropriate

[snoyberg]

@andrewthad Just to clarify on falling back, stack never does something different than what's in the config file. Fallback refers to when generating the config file itself.

@chreekat That table really helps, thank you! I realize this is bike shedding, but I'll do it anyway: setup still feels more correct to me than init, since it's not just initializing things but properly installing. For both cabal and ghc-pkg, init has a much cheaper implication. Can those in favor of init explain why they prefer it?

[chreekat]

I actually have no preference on setup vs init; I was just following the
"consensus" (which may or may not have been a single person). :)

On Wed, Jun 10, 2015 at 7:37 AM, Michael Snoyman notifications@github.com
wrote:

@andrewthad https://github.com/andrewthad Just to clarify on falling
back, stack never does something different than what's in the config file.
Fallback refers to when generating the config file itself.

@chreekat https://github.com/chreekat That table really helps, thank
you! I realize this is bike shedding, but I'll do it anyway: setup still
feels more correct to me than init, since it's not just initializing things
but properly installing. For both cabal and ghc-pkg, init has a much
cheaper implication. Can those in favor of init explain why they prefer it?

—
Reply to this email directly or view it on GitHub
#253 (comment)
.

[cschneid]

Until this issue is resolved, is there any guide / example of what a brand new project looks like? I don't mind placing & editing files manually while tooling is getting going. Or should I make a traditional cabal project and convert that somehow?

[borsboom]

I agree with keeping init for initializing the project. It's consistent with git, cabal, vagrant, yesod, and probably others I'm not thinking of right now.

[snoyberg]

There haven't been any comments for a few days, so I'll try to summarize the action items/deltas from current behavior:

• stack setup remains as-is, requiring a local stack.yaml and installing GHC
• stack new will be added, and will create a complete project with source code, .cabal, and stack.yaml files (covered by stack new for creating new projects with templates #137)
• stack build will no longer automatically create a stack.yaml file. Instead, if it sees a .cabal file in the current directory or subdirectory, it will recommend stack init. Otherwise, it will recommend stack new
• stack init will take a number of flags to control how it should behave regarding preferences for snapshots, and perhaps will accept user input to control what it does (thoughts on the latter?)
• stack config can be added to modify an existing stack.yaml in standard ways, like adding new packages, updating extra-deps, changing the resolver, etc. (should be handled under a different issue, whoever's interested in this behavior should open such an issue)

If there are no objections, let's move ahead with implementing this.

[mboes]

LGTM.

Any interactive user input should be providable from the command line too, for scriptability.

[snoyberg]

Agreed, good point.

[rvion]

I think by default, it would be usefull to have stack new and stack init populate all possible stack.yml fields with default values, so it's transparent for people.

[snoyberg]

OK, this is implemented, feedback welcome. The summary is that stack init exists, checks all .cabal files in the current directory and subdirs, and takes the arguments --fallback, --resolver, --prefer-nightly, and --prefer-lts. Here's a sample session:

$ stack unpack text
Unpacked text-1.2.1.1 to /home/vagrant/Desktop/text-1.2.1.1/
$ cd text-1.2.1.1/
$ stack build
Unable to find a stack.yaml file in the current directory (/home/vagrant/Desktop/text-1.2.1.1/) or its ancestors
Recommended action: stack init
$ stack init
Writing default config file to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
Basing on cabal files:
- /home/vagrant/Desktop/text-1.2.1.1/text.cabal
- /home/vagrant/Desktop/text-1.2.1.1/tests/text-tests.cabal
- /home/vagrant/Desktop/text-1.2.1.1/benchmarks/text-benchmarks.cabal

Checking against build plan lts-2.14
Selected resolver: lts-2.14
Wrote project config to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
$ stack init
Refusing to overwrite existing stack.yaml, please delete before running stack init
$ rm -f stack.yaml && stack init --prefer-nightly
Writing default config file to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
Basing on cabal files:
- /home/vagrant/Desktop/text-1.2.1.1/text.cabal
- /home/vagrant/Desktop/text-1.2.1.1/tests/text-tests.cabal
- /home/vagrant/Desktop/text-1.2.1.1/benchmarks/text-benchmarks.cabal

Checking against build plan nightly-2015-06-16
Selected resolver: nightly-2015-06-16
Wrote project config to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml

And then the failure cases. I modified text.cabal to be incompatible with LTSes and NIghtlies:

$ rm -f stack.yaml && stack init
Writing default config file to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
Basing on cabal files:
- /home/vagrant/Desktop/text-1.2.1.1/text.cabal
- /home/vagrant/Desktop/text-1.2.1.1/tests/text-tests.cabal
- /home/vagrant/Desktop/text-1.2.1.1/benchmarks/text-benchmarks.cabal

Checking against build plan lts-2.14
Checking against build plan lts-2.13
Checking against build plan nightly-2015-06-16
Checking against build plan nightly-2015-06-15
Checking against build plan lts-1.15
There was no snapshot found that matched the package bounds in your .cabal files.
Please choose one of the following commands to get started.

    stack init --resolver lts-2.14
    stack init --resolver lts-2.13
    stack init --resolver nightly-2015-06-16
    stack init --resolver nightly-2015-06-15
    stack init --resolver lts-1.15

You'll then need to add some extra-deps. See:

    https://github.com/commercialhaskell/stack/wiki/stack.yaml#extra-deps

Note that this will be improved in the future, see:

    https://github.com/commercialhaskell/stack/issues/116
$ rm -f stack.yaml && stack init --fallback lts-2.14
Writing default config file to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
Basing on cabal files:
- /home/vagrant/Desktop/text-1.2.1.1/text.cabal
- /home/vagrant/Desktop/text-1.2.1.1/tests/text-tests.cabal
- /home/vagrant/Desktop/text-1.2.1.1/benchmarks/text-benchmarks.cabal

Checking against build plan lts-2.14
Checking against build plan lts-2.13
Checking against build plan nightly-2015-06-16
Checking against build plan nightly-2015-06-15
Checking against build plan lts-1.15
Selected resolver: lts-2.14
Wrote project config to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
$ rm -f stack.yaml && stack init --resolver lts-2.14
Writing default config file to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml
Basing on cabal files:
- /home/vagrant/Desktop/text-1.2.1.1/text.cabal
- /home/vagrant/Desktop/text-1.2.1.1/tests/text-tests.cabal
- /home/vagrant/Desktop/text-1.2.1.1/benchmarks/text-benchmarks.cabal

Checking against build plan lts-2.14
Selected resolver: lts-2.14
Wrote project config to: /home/vagrant/Desktop/text-1.2.1.1/stack.yaml

If I don't hear any objections, I'll close this tomorrow.

[cschneid]

Does this address the bullet point from up top: Start a brand new project without any source code preexisting?

[snoyberg]

No, that's being covered in #137 (input would be welcome over there, we're talking out how it should work right now).