WIP: Dotnet new template #1989

kblohm · 2018-06-08T21:25:52Z

Hi,
i created a template to make it easier to get started using Fake.
I would appreciate some feedback if you are interested in this in general.
Some stuff to maybe discuss:

naming of parameters
available options
should this be in this repo at all?

Somewhat related:
What do u think about a Fake.All metapackage, similar to the AspNetCore.All? That would make it easier to get started. I personally do not really care about a few more megabytes to download to run my build. After the script is pretty much finished i can still clean up if i feel its getting too big. I am also kind of unsure what to include in a template like this, and a Fake.All package would get rid of that problem.

matthid · 2018-06-09T14:10:10Z

Yes Metapackages is something that came up very early and yes it is not clear at all what Fake.All would include. This is even true if we want to bundle some areas like Fake.DotNet into a metapackage. Would we include NuGet AND Paket for example or only give a opinionated set?

I guess we would give some opinionated commonly used packages.

matthid · 2018-06-09T14:11:53Z

Regarding the change itself: Yes I'd like to see a template. Can we add some documentation around this? (in particular for me to see how this works ;) )

kblohm · 2018-06-09T15:08:46Z

Would we include NuGet AND Paket for example or only give a opinionated set?

Do you mean the modules?
I would include all the modules inside a Fake.All package. Pretty much everything but the runtime. And if there is demand, create more dedicated metapackages, for example everything in Fake.DotNet.

* Update documentation in template.json

kblohm · 2018-06-09T16:39:02Z

If you want to try it locally (and dont want to build stuff yourself) i can also upload the nupkg.

matthid · 2018-06-09T16:47:18Z

No worries. I'll probably just merge straight to staging :)
Just something about the scripts. Will they all be in the created "project"? Should we have some defaults or ask the user if he wants scripts at all (and if yes which one)? Is this even possible?
Also if we decide to include all should we name them fake.sh/fake.cmd to clarify that they are executing fake?

matthid · 2018-06-09T16:49:42Z

Also out of curiosity: Can you use the template on top of an existing project?

kblohm · 2018-06-09T17:02:51Z

The build.fsx will always be created (you can specify the name though). Everything else is optional. I can also make the script optional.
I can rename build.sh and build.cmd to fake.*. I just used build because that is used in all the bootstrapping examples. But fake.* might make more sense.
And yes you can use it on top of an existing project. All the dotnet new stuff will just abort if it would have to override a file and i am pretty sure you can not merge stuff into existing files. So if you have a project with a paket.dependencies for example, you would have to usedotnet new fake --dependencies none so no new file is created (or dotnet new --force fake to override).
It does also not create any project or solution files (expect the build.proj, if that is the prefered way to bootstrap). That has the advantage that you can use other default templates and also use it for C#.
I could however create a second template that creates a solution and an example project and maybe also a build-script that does some actual stuff, like generating documentation. Then we would need to come up with more names for templates.

matthid · 2018-06-09T22:23:58Z

because that is used in all the bootstrapping examples.

I'd say this is for historic reasons, maybe we should change there as well?

Anyway, very nice idea thanks for looking into that. Should I merge as is? I think it looks quite usable already.
We can figure out the best defaults and namings along the way. There are probably quite some corner cases like maybe use the template to create the scripts (without overwriting build.fsx), but we don't have to support everything at start.

* Fix renaming of build.fsx

kblohm · 2018-06-10T06:50:15Z

I renamed the files, should be OK now.
Thanks for the feedback.
Two more things:

In your opinion, if i want to make a template with more then just a basic build setup, should i contribute to ProjectScaffold or should i add a second one here?
If you have a unix system, would you mind testing the fake.sh file? Once for dotnet new fake --bootstrap project and once for dotnet new fake --bootstrap tool? I am not sure if they actually work, i could only test with git-bash.

matthid · 2018-06-10T09:15:47Z

In your opinion, if i want to make a template with more then just a basic build setup, should i contribute to ProjectScaffold or should i add a second one here?

Uah, I believe the community is kind of split around that. My personal opinion is that is has been superseded by dotnet sdk templates. What's left is a centralized place to get started with community provided setup (Paket, Fake, Expecto, Unquote, FSharp.Formatting, everything put together ready to use). So what's missing is a ProjectScaffold-like template and a place for the documentation. (Which in the end could just be the current ProjectScaffold repository and homepage). fsprojects/ProjectScaffold#324

ProjectScaffold could also just be a script executing the various templates around the ecosystem:
Ie. it executes this template and some others to create various projects (testing, hello world, fake, paket, fsharp.formatting) and then writes a fake script to integrate everything. No idea if this is a good idea in practice :)

I guess everyone with a strong opinion there can have a huge impact, so feel free to just send PRs for what you feel is a good way forward (open issues if you feel it could be controversial)

If you have a unix system, would you mind testing the fake.sh file? Once for dotnet new fake --bootstrap project and once for dotnet new fake --bootstrap tool? I am not sure if they actually work, i could only test with git-bash.

As I'm on windows as well we can either add a integration test around that or I can try to use windows subsystem for windows :). Will merge on staging and take a look.

baronfel · 2018-06-10T11:32:35Z

I can test the Unix version of things. Could you provide a little detail about what exactly I should try out?

matthid · 2018-06-10T11:33:54Z

@baronfel I'll report back to you when we have fixed staging but then it's basically:

dotnet new --nuget-source https://www.myget.org/F/fake-vsts/api/v3/index.json -i "fake-template::*"
dotnet new fake --bootstrap project
dotnet new fake --bootstrap tool

matthid · 2018-06-10T11:36:30Z

build.fsx

@@ -1099,6 +1099,17 @@ Target.create "_DotNetPackage" (fun _ ->
                else c.Common
        } |> dtntSmpl) "Fake.sln"

+    // pack template
+    let templateProj = appDir @@ "fake-template"
+    DotNet.pack (fun c ->


Shouldn't we just add it to Fake.sln?

kblohm · 2018-06-10T11:36:51Z

@matthid
This:

let netCoreProjs =
!! (appDir </> "*/*.fsproj")

is probably the problem. The template is just a .proj file. I did also not add the template to the Fake.sln. dotnet sln add did not like that.

matthid · 2018-06-10T11:40:09Z

@kblohm thanks didn't noticed, will try to fix :)

kblohm · 2018-06-10T11:40:19Z

Yes it would be nice to add it to the solution file. That would probably not fix the deployment though, as

Target.create "DotNetCorePushNuGet" (fun _ ->
    // dotnet pack
    netCoreProjs
    -- (appDir </> "Fake.netcore/*.fsproj")
    |> Seq.iter(fun proj ->
        let projName = Path.GetFileName(Path.GetDirectoryName proj)
        !! (sprintf "nuget/dotnetcore/%s.*.nupkg" projName)
        -- (sprintf "nuget/dotnetcore/%s.*.symbols.nupkg" projName)
        |> Seq.iter (nugetPush 4))
)

would still not push it to nuget.
I am not really sure why dotnet sln add does not want me to add a .proj file.
We could probably just change it to fsproj.

matthid · 2018-06-10T11:40:24Z

dotnet sln add did not like that.

Ok understood

kblohm · 2018-06-10T11:44:29Z

I just had a look at https://github.com/aspnet/templating/tree/dev/src/Microsoft.DotNet.Web.ProjectTemplates. They are also just using .csproj files for their template projects. So the easiest way might be to just change it to fsproj and add it to the sln file. Then we could also remove the special handling.

matthid · 2018-06-10T11:49:51Z

haha, probably for the very same reason :) Will try

matthid · 2018-06-10T14:56:58Z

@baronfel Please test with dotnet new --nuget-source https://www.myget.org/F/fake-vsts/api/v3/index.json -i "fake-template::5.1.0-alpha*"

baronfel · 2018-06-10T15:03:29Z

as I expected you need to set the executable flag on the .sh script. This can be done with post-install actions, MiniScaffold does it like this: https://github.com/TheAngryByrd/MiniScaffold/blob/master/Content/.template.config/template.json#L19

baronfel · 2018-06-10T15:05:03Z

The unique ids for the postactions that can be run is here: https://github.com/dotnet/templating/wiki/Post-Action-Registry, but I think you'll just need that exact code block from miniscaffold.

kblohm · 2018-06-10T15:06:39Z

It should mainly be dotnet new fake --bootstrap tool. Because in fsprojects/FSharp.Control.Reactive#110 i did not manage to get this to run on travis and MAYBE its because of build.sh.
I am also not sure if the script can be run twice, because you have to check if FAKE is already installed, otherwise dotnet tool install will be angry.

matthid · 2018-06-10T15:10:33Z

Now after catching this I think we need an integration test which ensures that dotnet new -i && dotnet new fake && ./fake.sh build works without error

baronfel · 2018-06-10T15:11:16Z

is there a way to make the restore in fake.sh a one-time thing? it introduces a noticable amount of latency in using the script:

➜  fake-project git:(master) ✗ ./fake.sh
  Restore completed in 28.85 ms for /Users/chethusk/oss/fake-project/build.proj.
  Restore completed in 50.92 ms for /Users/chethusk/oss/fake-project/build.proj.

real	0m5.334s
user	0m4.109s
sys	0m0.897s

baronfel · 2018-06-10T15:12:08Z

@matthid you can again look at MiniScaffold for how to set up such a test easily. The test suite there installs the new packed template locally and tests it.

matthid · 2018-06-10T15:12:44Z

@baronfel This really is a .net sdk issue because it should do an implicit restore (I believe there is an open issue for that). Once that is fixed we can remove the explicit restore and let the sdk handle that.

matthid · 2018-06-10T15:14:21Z

@baronfel Yes my thinking was that dotnet new --nuget-source /full/path/to/dir/containing/nupkg -i "fake-template::<version>" would work

In any case can you or @kblohm send a PR to add the test?

baronfel · 2018-06-10T15:14:30Z

also if the user specifies a non-default script name I think the generated sh/cmd scripts should specify the provided name in the invocations, otherwise the new-ly created script name will not be run.

This is made harder by the fact that run and build have different formats for where the script file goes, so you'd almost have to do parsing of the first argument to see if it is build or run and change things based on that.

baronfel · 2018-06-10T15:14:55Z

sorry for lots of comments, just noting things as I see them

baronfel · 2018-06-10T15:20:06Z

installing a global tool to a subdir doesn't seem to work:

➜  fake-tool git:(master) ✗ ./fake.sh
You can invoke the tool using the following command: fake
Tool 'fake-cli' (version '5.0.0') was successfully installed.
The application to execute does not exist: '/Users/chethusk/oss/fake-tool/.fake/..//Users/chethusk/oss/fake-tool/.fake/.store/fake-cli/5.0.0/fake-cli/5.0.0/tools/netcoreapp2.1/any/fake-cli.dll'.

The binary gets placed in .fake/fake, but that binary has a path hard-coded into it that is not correct:

/Users/chethusk/oss/fake-tool/.fake/..//Users/chethusk/oss/fake-tool/.fake/.store/fake-cli/5.0.0/fake-cli/5.0.0/tools/netcoreapp2.1/any/fake-cli.dll

Note that the partially-mangled correct path is findable there: /Users/chethusk/oss/fake-tool/.fake/.store/fake-cli/5.0.0/fake-cli/5.0.0/tools/netcoreapp2.1/any/fake-cli.dll

matthid · 2018-06-10T15:20:13Z

also if the user specifies a non-default script name I think the generated sh/cmd scripts should specify the provided name in the invocations, otherwise the new-ly created script name will not be run.

Currently the scripts will just wrap fake and not actually build a target (which I think makes sense).

This is made harder by the fact that run and build have different formats for where the script file goes,

My opinion on this is that if people use a non-default script-name fake build will still work (as long as there is only ONE script). On the other side if people use something else than build.fsx they probably should know what they are doing

sorry for lots of comments, just noting things as I see them

No worries there, perfectly reasonable ;)

baronfel · 2018-06-10T15:21:51Z

There's also a bug with install when dotnet cli is in a non-standard location: https://github.com/dotnet/cli/issues/9114

baronfel · 2018-06-10T15:28:27Z

@matthid working on a PR now for chmod on the scripts + a test run

kblohm · 2018-06-10T15:56:07Z

@baronfel thanks for looking into this. All that fancy linux stuff and not being able to test locally makes this kinda hard.

baronfel · 2018-06-10T15:59:48Z

happy to help :)

kblohm · 2018-06-10T16:14:36Z

So until dotnet/cli#9114 is fixed, you have to use the build.proj approach to run on unix, right? Or am i missing something in that issue?

baronfel · 2018-06-10T17:10:30Z

well, you can also use the global tool, which is what I plan on standardizing on. should we make that a template-option? --bootstrap global-tool? and then skip the tool installation in the shell/cmd scripts?

kblohm · 2018-06-10T17:17:11Z

That is what --boostrap none does.
Or do you mean not generate anything AND install the tool globally?

kblohm · 2018-06-10T20:10:35Z

@baronfel

as I expected you need to set the executable flag on the .sh script.

So i can not create this on windows for you to run? That pretty much means whoever does the bootstrapping has to be on unix? What happens otherwise?

matthid · 2018-06-10T20:14:09Z

That pretty much means whoever does the bootstrapping has to be on unix? What happens otherwise?

Uah interesting point. On windows we would need to run git update-index --chmod=+x fake.sh but we don't even know if git is available and if it is if the user created a git repository already...

baronfel · 2018-06-10T20:17:16Z

Yes but this is a problem we would have had even with the change-permissions activity. The BCL File Metadata structures didn't until recently have the capacity to even report file permissions, so it's taking even longer for that ability to trickle down into all the tools. It's very irritating.

kblohm · 2018-06-10T20:29:09Z

Ahh! Thanks @matthid that helped! So yeah i can speak from experience now, that is a problem when you create the files on windows and try to build on unix.

matthid · 2018-06-10T20:31:36Z

Not sure if we can somehow build that into the template (it would be nice if we find a way). But if not it is not a huge deal chmod errors are quite common so I hope users are able to figure it out.

Dotnet new template

bcf74d8

matthid changed the base branch from master to release/next June 9, 2018 13:15

* Add documentation for fake-template

0373508

* Update documentation in template.json

* Rename build.sh and build.cmd to fake.sh and fake.cmd

52bb386

* Fix renaming of build.fsx

matthid merged commit 3f24a8f into fsprojects:release/next Jun 10, 2018

kblohm deleted the fakeTemplate branch June 10, 2018 11:11

matthid reviewed Jun 10, 2018

View reviewed changes

kblohm mentioned this pull request Jun 10, 2018

Disussion: Relation/Advantage over dotnet sdk templates? fsprojects/ProjectScaffold#324

Closed

baronfel mentioned this pull request Jul 2, 2018

fake-cli 'tool path' install doesn’t seem to work on OS X/linux #2013

Closed

WIP: Dotnet new template #1989

WIP: Dotnet new template #1989

Conversation

kblohm commented Jun 8, 2018 • edited Loading

matthid commented Jun 9, 2018

matthid commented Jun 9, 2018

kblohm commented Jun 9, 2018

kblohm commented Jun 9, 2018

matthid commented Jun 9, 2018

matthid commented Jun 9, 2018

kblohm commented Jun 9, 2018

matthid commented Jun 9, 2018

kblohm commented Jun 10, 2018 • edited Loading

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

matthid commented Jun 10, 2018

matthid Jun 10, 2018

Choose a reason for hiding this comment

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

baronfel commented Jun 10, 2018

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

baronfel commented Jun 10, 2018

matthid commented Jun 10, 2018

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

baronfel commented Jun 10, 2018

baronfel commented Jun 10, 2018

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

baronfel commented Jun 10, 2018

kblohm commented Jun 10, 2018

baronfel commented Jun 10, 2018

kblohm commented Jun 10, 2018

baronfel commented Jun 10, 2018

kblohm commented Jun 10, 2018 • edited Loading

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

baronfel commented Jun 10, 2018

kblohm commented Jun 10, 2018

matthid commented Jun 10, 2018

kblohm commented Jun 8, 2018 •

edited

Loading

kblohm commented Jun 10, 2018 •

edited

Loading

kblohm commented Jun 10, 2018 •

edited

Loading