Package Management for FSI #5850

KevinRansom · 2018-10-31T19:32:29Z

This is heavily based on the work @forki and @dsyme and others did for this PR: #2483
Thanks guys, that is a genius mechanism.

Notes:

It's a WIP --- there are undoubtedly a ton of bugs, we need to work through the scenarios. I expect there are places where Don and I will have philosophical differences, we usually do :-)
It works on the desktop with fsi.exe and on the coreclr, and in VS. We rely on the dotnet sdk for resolution, essentially we generate a .fsproj and restore it for resolution.
We surface dotnet/msbuild diagnostic messages, using a verbosity filter.
The PM integration point calls the package resolution mechanism with a list of all of the resolution strings specified in the session, which then computes the assemblies to reference.
If resolution fails the last group of references added are removed from the list of resolution strings, after all it made the graph non resolvable. The developer can retype a corrected command.
It is pluggable, the approach and interfaces are very close to the original PR, so I expect Steffen, can rework his paket resolver pretty quick.
ToDo: Currently it doesn't allow a pluggable resolver to be referenced using #r"nuget:" that is my next thing to get working.
This adds a --compilertools: command line option for specifying the path to the root of a nuget package containing a compiler tool, such as a package manager. The package manager is then resolved similarly to type providers, with fsc and fsi computing a run time version and then compatible framework ids to probe for a dll to load.
ToDo: compile single script file doesn't seem to work, since I expected it to … I am slightly confused.
ToDo unify TP and DependencyManager framework detection … probably publicly expose it from FCS.
The Fsharp.Data package works on coreclr and desktop, so this script does what you might expect:
There are some basic tests but many more to do.

How to see it work:
What I do:

With the branch on a machine, build the F# VisualFSharp tools by:

build vs

Run vs by:
devenv /RootSuffix RoslynDev

Create a new script file and start typing:

For example:
Note that openweathermap which is very cool now requires an APIKey, the number of access is quiet limited, so you may get an error back from the message, so, best to grab an api key of your own, then other folk looking at it won't use up your experiment time.

#r "nuget:include=fsharp.data, version=3.0.0"

open FSharp.Data

// Api Key = 93a2637fe6dad8426d128d5289325ca9
open FSharp.Data

type Weather = JsonProvider<"http://api.openweathermap.org/data/2.5/weather?id=2172797&APPID=93a2637fe6dad8426d128d5289325ca9">
let apiUrl = "http://api.openweathermap.org/data/2.5/weather?id=2172797"
let sf = Weather.Load(apiUrl + "&APPID=93a2637fe6dad8426d128d5289325ca9")
let country = sf.Sys.Country
let speed = sf.Wind.Speed
let temp = sf.Main.Temp
printfn "Temp for Australia is {%M}" temp

Note the intellisense and syntax colouring:

Select everything and send to interactive:


Microsoft (R) F# Interactive version 10.2.3 for F# 4.5
Copyright (c) Microsoft Corporation. All Rights Reserved.

For help type #help;;

> 
"C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\FSharp\Microsoft.FSharp.NetSdk.targets" cannot be imported again. It was already imported at "c:\kevinransom\visualfsharp\Tools\dotnet20\sdk\2.1.403\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FSharpTargetsShim.targets (63,3)". This is most likely a build authoring error. This subsequent import will be ignored. [C:\Users\kevinr\AppData\Local\Temp\uy2ymaj3.jz2\Project.fsproj]
visiting C:\Users\kevinr\AppData\Local\Temp\uy2ymaj3.jz2\Project.fsproj.fsx
yielding source C:\Users\kevinr\AppData\Local\Temp\uy2ymaj3.jz2\Project.fsproj.fsx
[Loading C:\Users\kevinr\AppData\Local\Temp\uy2ymaj3.jz2\Project.fsproj.fsx]
namespace FSI_0002.Project

Temp for Australia is {298.15}
type Weather = FSharp.Data.JsonProvider<...>
val apiUrl : string =
  "http://api.openweathermap.org/data/2.5/weather?id=2172797"
val sf : FSharp.Data.JsonProvider<...>.Root =
  {
  "coord": {
    "lon": 145.77,
    "lat": -16.92
  },
  "weather": [
    {
      "id": 802,
      "main": "Clouds",
      "description": "scattered clouds",
      "icon": "03n"
    }
  ],
  "base": "stations",
  "main": {
    "temp": 298.15,
    "pressure": 1015,
    "humidity": 78,
    "temp_min": 298.15,
    "temp_max": 298.15
  },
  "visibility": 10000,
  "wind": {
    "speed": 3.1,
    "deg": 140
  },
  "clouds": {
    "all": 40
  },
  "dt": 1541012400,
  "sys": {
  ...
val country : string = "AU"
val speed : decimal = 3.1M
val temp : decimal = 298.15M
val it : unit = ()

>

KevinRansom · 2018-10-31T19:58:29Z

Since many of you are bound to ask : how to test the coreclr version

This is one way that I use:

in the visualfsharp repo:
build coreclr
Sync the dotnet cli open source rebuild from here: https://github.com/dotnet/cli
checkout the branch release/2.1.5xx
in the directory: src\tool_fsharp add the file named Nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>  
    <add key="artifacts" value="{path to your visualfsharp repo --- in my case: C:\kevinransom\visualfsharp}\artifacts" />
  </packageSources>
</configuration>

Edit the file : build\DependencyVersions.props

Change this:
    <MicrosoftFSharpCompilerPackageVersion>10.2.3-rtm-181017-0</MicrosoftFSharpCompilerPackageVersion>

To a value that matches the most recent build in your artifacts directory
E.g.
    <MicrosoftFSharpCompilerPackageVersion>10.2.3-rtm-181031-00</MicrosoftFSharpCompilerPackageVersion>

Then type build:

building/whiring and test running will then ensue. If build fails and the failure is not to do with the fsc_tools project, then don't worry, a testable dotnet cli has been created in:

bin\2\win-x64\dotnet


c:\kevinransom\visualfsharp>C:\kevinransom\cli\bin\2\win-x64\dotnet\dotnet.exe C:\kevinransom\cli\bin\2\win-x64\dotnet\sdk\2.2.200-preview-009554\FSharp\fsi.exe

Microsoft (R) F# Interactive version 10.2.3 for F# 4.5
Copyright (c) Microsoft Corporation. All Rights Reserved.

For help type #help;;

> #quit;;

c:\kevinransom\visualfsharp>C:\kevinransom\cli\bin\2\win-x64\dotnet\dotnet.exe C:\kevinransom\cli\bin\2\win-x64\dotnet\sdk\2.2.200-preview-009554\FSharp\fsi.exe

Microsoft (R) F# Interactive version 10.2.3 for F# 4.5
Copyright (c) Microsoft Corporation. All Rights Reserved.

For help type #help;;

> #r "nuget:include=fsharp.data, version=3.0.0"

open FSharp.Data

// Api Key = 93a2637fe6dad8426d128d5289325ca9

open FSharp.Data

type Weather = JsonProvider<"http://api.openweathermap.org/data/2.5/weather?id=2172797&APPID=93a2637fe6dad8426d128d5289325ca9">

let apiUrl = "http://api.openweathermap.org/data/2.5/weather?id=2172797"

let sf = Weather.Load(apiUrl + "&APPID=93a2637fe6dad8426d128d5289325ca9")
let country = sf.Sys.Country
let speed = sf.Wind.Speed
let temp = sf.Main.Temp

printfn "Temp for Australia is {%M}" temp

;;
visiting C:\Users\kevinr\AppData\Local\Temp\t2duao4m.sby\Project.fsproj.fsx
yielding source C:\Users\kevinr\AppData\Local\Temp\t2duao4m.sby\Project.fsproj.fsx
[Loading C:\Users\kevinr\AppData\Local\Temp\t2duao4m.sby\Project.fsproj.fsx]
namespace FSI_0002.Project

Temp for Australia is {298.15}
type Weather = FSharp.Data.JsonProvider<...>
val apiUrl : string =
  "http://api.openweathermap.org/data/2.5/weather?id=2172797"
val sf : FSharp.Data.JsonProvider<...>.Root =
  {
  "coord": {
    "lon": 145.77,
    "lat": -16.92
  },
  "weather": [
    {
      "id": 802,
      "main": "Clouds",
      "description": "scattered clouds",
      "icon": "03n"
    }
  ],
  "base": "stations",
  "main": {
    "temp": 298.15,
    "pressure": 1015,
    "humidity": 78,
    "temp_min": 298.15,
    "temp_max": 298.15
  },
  "visibility": 10000,
  "wind": {
    "speed": 3.6,
    "deg": 140
  },
  "clouds": {
    "all": 40
  },
  "dt": 1541014200,
  "sys": {
  ...
val country : string = "AU"
val speed : decimal = 3.6M
val temp : decimal = 298.15M
val it : unit = ()

> #quit;;

c

src/buildfromsource/FSharp.Compiler.Private/FSComp.resx

cartermp · 2018-10-31T20:36:36Z

Link #2407

dsyme · 2018-10-31T23:42:25Z

I scanned the basic shape of the PR and couldn't find major philosophical differences :) It looks great!

forki · 2018-11-01T07:43:30Z

My biggest question is about bootstrapping? How can we make sure F# users have paket access ootb when they are installing dotnet core. This whole thing is only interesting if we can get this to work. Am Do., 1. Nov. 2018, 00:42 hat Don Syme <notifications@github.com> geschrieben:

…

I scanned the basic shape of the PR and couldn't find major philosophical differences :) It looks great! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5850 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNLAAdVr1QsZwQVW97TLTUYY5gEKkks5uqjVigaJpZM4YE0CS> .

dsyme · 2018-11-01T14:42:07Z

My biggest question is about bootstrapping? How can we make sure F# users have paket access ootb when they are installing dotnet core. This whole thing is only interesting if we can get this to work.

Maybe it can happen, do you have an idea how you might want this to work?

It does feel like Paket may need some bootstrap script, user-global paket install or something.

forki · 2018-11-01T16:29:28Z

There are multiple ways to do it

assume people already have global paket installed
put a bootstrapper into the box
allow to curl paket (or any other #r plugin) with a one line #r directive and resolve in multiple phases

cartermp · 2018-11-01T16:52:13Z

I don't think we can put a bootstrapper in the box unless we authored and owned a paket bootstrapper that was subject to all the same requirements that any MS-authored component has.

But in general, since this is pluggable, it stands to reason that any other installed tool could also supply a plugin. That could be paket or fake. But @KevinRansom knows more about how it actually plugs in, and this also needs to be documented. But we're a ways off from this being ship-ready anyways.

forki · 2018-11-01T17:37:39Z

Well I already contributed that bootstrapper and it wouldn't be to hard to do for you from scratch. But I know it's always a hard thing to do for Microsoft, so that's why we came up with the generic way to load #r providers in my original PR. Am Do., 1. Nov. 2018, 18:03 hat Phillip Carter <notifications@github.com> geschrieben:

…

I don't think we can put a bootstrapper in the box unless we authored and owned a paket bootstrapper that was subject to all the same requirements that any MS-authored component has. But in general, since this is pluggable, it stands to reason that any other installed tool could also supply a plugin. That could be paket or fake. But @KevinRansom <https://github.com/KevinRansom> knows more about how it actually plugs in, and this also needs to be documented. But we're a ways off from this being ship-ready anyways. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5850 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgND1lWgoWXOOWHUktX3Gw7qPMXCLkks5uqylkgaJpZM4YE0CS> .

KevinRansom · 2018-11-01T18:15:13Z

@forki, we will do whatever we must to make a paket packagemanager usable and work well.

Either:
A nuget package containing a paket manager will be reference-able and usable immediately in the same script.

Or we will ship an in-box bootstrapping package manager that will go fetch the paket package manager ideally on first use of #r "paket:blah ...."
or with a #r "compilertool:paket.packagemanagerId, version=1.0.0"
or something

We are committed to make it work and work well. TBH: if paket doesn't work well, Don would have my guts for garters.

Kevin

cartermp · 2018-11-01T20:15:45Z

Usability note from today:

You must turn off shadow copying for assemblies in Tools > Options > F# Tools > F# Interactive for this to execute in F# interactive.

That makes things difficult, since unless we can resolve the design to not require that be off, we'll either have a "doesn't work by default" experience or a "break some people" experience.

cartermp · 2018-11-01T20:32:45Z

FYI this currently breaks all editor tooling except for a standalone F# script file.

KevinRansom · 2018-11-01T21:26:14Z

Cab you explain the all editor tooling? Only fsi.exe turns on shadow copy and turning that off doesn't break anything as far as I know. Except of course for FSI now locking referenced assemblies.

cartermp · 2018-11-01T22:34:58Z

File > New Project > any F# project

KevinRansom · 2018-11-01T22:49:22Z

Is this what you are seeing?

if so .fs are not intended to support #r directives.

If the project contains a .fsx file, as the bottom file, then it is expected to work

cartermp · 2018-11-01T22:57:04Z

No, in the build of master against 15.8.8 with this VSIX installed, no files have language service lightup. What build are you testing against?

KevinRansom · 2018-11-01T22:59:20Z

Dogfood (15.9 preview 5). Although I can't imagine why 15.8.8 would be a problem. I'll take a look.

KevinRansom · 2018-11-01T23:09:42Z

Works on my box, I will take a look at your machine tomorrow:

cartermp · 2018-11-02T00:11:23Z

src/fsharp/FSharp.DependencyManager/FSharp.DependencyManager.fs

+                            validatePackageName v "mscorlib"                                                    // Even though it's not a real package.
+                            validatePackageName v "FSharp.Core"
+                            validatePackageName v "System.ValueTuple"
+                            validatePackageName v "NETStandard.Library"


We may want to consider Microsoft.NETCore.App and/or whatever gives us access to Span. I suppose that's a bit tricky, but Iactually just saw some requests about how to use Span with FSI, and the only way to do that here would be to reference the relevant bits.

enricosada · 2018-11-02T11:24:46Z

src/fsharp/FSharp.DependencyManager/FSharp.DependencyManager.fs

+                    raise (ArgumentException(sprintf "PackageManager can not reference the System Package '%s'" packageName))               // @@@@@@@@@@@@@@@@@@@@@@@ Globalize me please
+
+            let mutable loggerVerbosity = ""
+            let references = [


@KevinRansom parsing the console log to read the dependencies can be improved.

In fsac/ionide/fable etc, to get the fsc args (so similar to this) we invoke an external msbuild (like here), and to do so we use https://github.com/enricosada/dotnet-proj-info/

That lib/tool invoke an external msbuild, but instead of parsing the log use a different strategy.

To the fsproj is added a target file, with a target who run in design mode, and save the items to file (like a csv).

the target is like https://github.com/enricosada/dotnet-proj-info/blob/master/src/dotnet-proj-info/Inspect.fs#L170-L196 so you can directly use that if you want

The trick is good because there is no parsing of log and work with any fsproj/csproj, and you can read everything you want inside the msbuild target (at any msbuild step, with a single invocation) and in the target write the output file as you wish

Some features: get installed .net frameworks, get fsc args from proj or fsx, get props/items, etc. Will use same design time run, like vs, so will not invoke fsc either.
Works the same for.net core fsproj but also for verbose fsproj (with another trick, replace completly projectcracker), but here is not needed.

If you want, can be used as library (like fsac/fable does), so we can reuse the code too, the library (https://www.nuget.org/packages/Dotnet.ProjInfo/) doesnt have any deps, only fsharp.core

Example use as lib: https://github.com/fable-compiler/Fable/blob/master/src/dotnet/Fable.Compiler/CLI/ProjectCoreCracker.fs#L66-L75
fsac usage is more complicated because does navigate the p2p hierachy too, and gather additional info, but here is not needed

happy to chat about it, ihmo can be a nice way to share and improve a common library and strategy.
Really happy to move it somewhere else (like in fsharp/dotnet org) and give you ownership too (plus doing things like strong name/etc if needed).

enricosada · 2018-11-02T11:41:17Z

i'll check how to bootstrap paket.

src/fsharp/FSharp.DependencyManager/FSharp.DependencyManager.Utilities.fs

KevinRansom · 2018-11-02T17:42:14Z

@enricosada the mono build shouldn't be under reshaped msbuild. And so this code path is the controlling one which uses msbuild inproc:
https://github.com/Microsoft/visualfsharp/pull/5850/files/0b62b3d6e9d6087d3cfc243d1afe825c5c49e852#diff-c3a4578a1255d318a537ddaee38842e5R256

cartermp · 2018-11-02T20:19:47Z

Just confirming that there was something environmental going on my end. This works just fine with 15.8.8.

Some usability notes:

Needing to specify the version explicitly, though due to a NuGet limitation, needs to get resolved. nuget: FSharp.Data should always give the latest stable version, for example. It should be similar to dotnet add package.
The editor for F# scripts is noticeably slow when using this. Likely a lot of redundant work going on.
Need to find a way not to rely on Shadow Copy being off. This is on by default for users, and turning it off would regress current workflows for people. This could end up being a showstopper.
Logging done in FSI eventually needs to be cleaned up, but it makes sense it see it now given that we're very early on.

forki · 2018-11-02T21:01:11Z

That shadow copy part is only applying to VS, right? Other editors and fsi standalone could still work fine? BTW you should also look at the current implementation of this whole thing in FAKE. We certainly don't want to break FAKE. /cc @matthid for details Am Fr., 2. Nov. 2018, 21:46 hat Phillip Carter <notifications@github.com> geschrieben:

…

Just confirming that there was something environmental going on my end. This works just fine with 15.8.8. Some usability notes: - Needing to specify the version explicitly, though due to a NuGet limitation, needs to get resolved. nuget: FSharp.Data should always give the latest stable version, for example. - The editor for F# scripts is noticeably slow when using this. Likely a lot of redundant work going on. - Need to find a way not to rely on Shadow Copy being off. This is on by default for users, and turning it off would regress current workflows for people. This could end up being a showstopper. - Logging done in FSI eventually needs to be cleaned up, but it makes sense it see it now given that we're very early on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5850 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNMHs0AvC0L4A3RMuuqbX98WFX1ZRks5urK8SgaJpZM4YE0CS> .

KevinRansom · 2018-11-02T22:06:27Z

Nope. VS doesn’t use shadowcopy, it’s just FSI that is the PITA. So FSI on the desktop will either have to choose shadowcopy or #r. However, I think we can run the msbuild stuff in a separate non-shadowcopying app domain, or out of process. Still plenty of options ahead of us. This would certainly not impact the paket provider since it won’t run msbuild in-process with FSI, which is the issue. It should be remembered that longer term the desktop framework is being de-emphasised so this is kind of moot. Anyway there From: Steffen Forkmann <notifications@github.com> Sent: Friday, November 2, 2018 2:10 PM To: Microsoft/visualfsharp <visualfsharp@noreply.github.com> Cc: Kevin Ransom <Kevin.Ransom@microsoft.com>; Mention <mention@noreply.github.com> Subject: Re: [Microsoft/visualfsharp] WIP --- Package Management for FSI (#5850) That shadow copy part is only applying to VS, right? Other editors and fsi standalone could still work fine? BTW you should also look at the current implementation of this whole thing in FAKE. We certainly don't want to break FAKE. /cc @matthid for details Am Fr., 2. Nov. 2018, 21:46 hat Phillip Carter <notifications@github.com<mailto:notifications@github.com>> geschrieben:

Just confirming that there was something environmental going on my end. This works just fine with 15.8.8. Some usability notes: - Needing to specify the version explicitly, though due to a NuGet limitation, needs to get resolved. nuget: FSharp.Data should always give the latest stable version, for example. - The editor for F# scripts is noticeably slow when using this. Likely a lot of redundant work going on. - Need to find a way not to rely on Shadow Copy being off. This is on by default for users, and turning it off would regress current workflows for people. This could end up being a showstopper. - Logging done in FSI eventually needs to be cleaned up, but it makes sense it see it now given that we're very early on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5850 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNMHs0AvC0L4A3RMuuqbX98WFX1ZRks5urK8SgaJpZM4YE0CS> .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgh.neting.cc%2FMicrosoft%2Fvisualfsharp%2Fpull%2F5850%23issuecomment-435507851&data=02%7C01%7CKevin.Ransom%40microsoft.com%7C88b046a3aa2348b9b9b408d64107951f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636767898169211059&sdata=c523JVR1ptisyenbSLVCbqY72pMGpn05dBZ2UybkEx4%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgh.neting.cc%2Fnotifications%2Funsubscribe-auth%2FAE76FiOKTbZE-9nr6fCwTGj3g2IS4wAyks5urLS2gaJpZM4YE0CS&data=02%7C01%7CKevin.Ransom%40microsoft.com%7C88b046a3aa2348b9b9b408d64107951f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636767898169221067&sdata=iF8nrrbdzog9cUx1fosfmKygDo19SqFFDw0B0zZVMt0%3D&reserved=0>.

matthid · 2018-11-02T23:56:08Z

Technically, this PR is a breaking change. For this to be compatible with fake all we need would be a public api in fcs. Ideally, no extra dll is required in that scenario...

KevinRansom · 2018-11-02T23:58:22Z

@matthid, what's broken exactly?

…to fsi-packetmanagement

goswinr · 2019-10-25T07:39:24Z

I am trying to us this already today. (for my own scripting editor)
I build the fsharp5 branch and tried this in fsiExe.exe:
How can I register a package manager key?

> #r "nuget:include=fsharp.data, version=3.0.0";;

  #r "nuget:include=fsharp.data, version=3.0.0";;
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

stdin(1,1): **error FS3216: Package manager key 'nuget' was not registered in** [D:\Git\fsharp\fsharp5\fcs\net461; D:\Git\fsharp\fsharp5\fcs\net461\], []. Currently registered:


  #r "nuget:include=fsharp.data, version=3.0.0";;
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

stdin(1,1): error FS3217: Processing of a script fragment has stopped because an exception has been raised

realvictorprm · 2019-10-25T08:26:27Z

See my tweet here with code example. Syntax should be #r: "nuget: PackageName".

https://twitter.com/realvictorprm/status/1173639408693723136?s=19

goswinr · 2019-10-25T12:49:24Z

@realvictorprm thanks! Your code example works. How can I use the FSharp.Compiler.Service.dll ?

I used the one from fsharp\artifacts\bin\fcs\net461 with

    let allArgs = [|"ignored" ; "--langversion:preview" ; "--noninteractive"|] 
    let fsiConfig = FsiEvaluationSession.GetDefaultConfiguration() 
    let fsiSession = FsiEvaluationSession.Create(fsiConfig, allArgs, inStream, textwriter, textwriter)

I still get this error.
error FS3216: Package manager key 'nuget' was not registered

KevinRansom · 2019-10-25T17:20:56Z

@Goswin, fcs need to deploy:
FSharp.DependencyManager.dll alongside the FSharp.Compiler.Private.dll

So use the assemblies built into FSharp.Compiler.Private.Scripting
fsharp\artifacts\bin\FSharp.Compiler.Private.Scripting

I will up0date the fcs build to do the necessary at some point.

KevinRansom · 2019-10-25T17:41:56Z

@Goswin, on coreclr, you should also use the latest version of FSharp.Data. Earlier versions don't work correctly due to a bug in the typeprovider sdk.

goswinr · 2019-10-25T20:02:51Z

@KevinRansom Thank you for you help!
After running the build command I only found fsharp\artifacts\bin\FSharp.Compiler.Private.Scripting\Debug\netstandard2.0
(I assume it is Ok to build net472 apps with those)
there are

FSharp.Compiler.Private.dll
FSharp.Compiler.Private.Scripting.dll
FSharp.Core.dll
FSharp.DependencyManager.dll

Shall i refrence those instead of the FSharp.Compiler.Service.dll ?
Do I also need to use this FSharp.Core.dll instead of the current nuget?

smoothdeveloper · 2020-01-16T07:43:57Z

src/fsharp/CompileOps.fs

+                            yield! resolveDependencyManagerSources filename
+#if DEBUG
+                            for (_,subFile) in sources do
+                               printfn "visiting %s - has subsource of %s " filename subFile


This and the other conditional enclosed printfn below fails the "load-test" test when run locally (in debug mode by default), could it be removed from master?

I think the answer to this is to adjust testing all-up, since these kinds of diagnostics are very important for environments like Jupyter (where this is enabled) to see what went wrong

Maybe a #if JUPYTER_NOTEBOOK_DEBUG is a good approach then? Not a big issue but wanted to report it may be an overlooked remnant.

The diagnostics could also be incorporated in the dependency manager itself (assuming jupyter will have its own) rather than CompileOps.fs.

@KevinRansom @dsyme was pointing at the printf here ^^^

smoothdeveloper

I'm giving this a shot reviving and adjusting the paket implementation (https://github.com/smoothdeveloper/Paket/tree/fsi-paket-dep-manager), this is working 🙂.

@brettfo / @KevinRansom is there a way to pull the version of master branch compiled fsi.exe out of continuous integration with a stable URI so I can work on integrating the tests in the CI pipeline in paket repository with all the tests we initially had in the experimental branch?

smoothdeveloper · 2020-01-26T21:29:44Z

src/fsharp/DependencyManager.Integration.fs

+            None)
+    |> Seq.filter (fun a -> ReflectionHelper.assemblyHasAttribute a dependencyManagerAttributeName)
+
+let registeredDependencyManagers = ref None


This reference is never written to, the assemblies and types seems to currently be scanned on each batch of dependency manager lines evaluated.

smoothdeveloper · 2020-01-26T21:36:46Z

src/fsharp/CompileOps.fs

+                            yield! resolveDependencyManagerSources filename
+#if DEBUG
+                            for (_,subFile) in sources do
+                               printfn "visiting %s - has subsource of %s " filename subFile


Maybe a #if JUPYTER_NOTEBOOK_DEBUG is a good approach then? Not a big issue but wanted to report it may be an overlooked remnant.

The diagnostics could also be incorporated in the dependency manager itself (assuming jupyter will have its own) rather than CompileOps.fs.

smoothdeveloper · 2020-01-26T21:37:11Z

src/fsharp/DependencyManager.Integration.fs

+    let keyProperty      = keyProperty.GetValue >> string
+
+    static member InstanceMaker (theType: System.Type, outputDir: string option) =
+        match ReflectionHelper.getAttributeNamed theType dependencyManagerAttributeName,


In the original code, those were nested checks; it looks much better that way, but I believe the first check (attribute on the type) should be handled separately, there are potentially lots of types that will be coming in the assemblies, better to reject them on not having the attribute, then do the remaining checks.

smoothdeveloper · 2020-01-26T21:41:45Z

src/fsharp/FSharp.DependencyManager/FSharp.DependencyManager.Utilities.fs

+namespace lib"
+
+    let generateProjectBody = @"
+<Project Sdk='Microsoft.NET.Sdk'>


Looks like msbuild black belt stuff 😄

smoothdeveloper · 2020-01-26T21:46:09Z

src/fsharp/DependencyManager.Integration.fs

+        with _ -> Seq.empty)
+    |> Seq.choose (fun path -> 
+        try
+            Some(AbstractIL.Internal.Library.Shim.FileSystem.AssemblyLoadFrom path)


The assemblies seem to be locked, would it be possible/better to shadow copy those?

smoothdeveloper · 2020-01-26T21:52:25Z

src/fsharp/DependencyManager.Integration.fs

+type internal IDependencyManagerProvider =
+    abstract Name : string
+    abstract Key: string
+    abstract ResolveDependencies : scriptDir: string * mainScriptName: string * scriptName: string * packageManagerTextLines: string seq * tfm: string -> bool * string list * string list


Consider replacing the string list with arrays or enumerables. this makes it possible to implement the provider without dependency on FSharp.Core.

This was a concern when I made the initial implementation of the provider interface so the same provider could be loaded from scriptcs (@glennblock), ideally the same interface could just work for C# interactive as well (given they implement similar hooks at some point).

The signature should be documented, it is not obvious that return type is:

success: bool

load scripts: string []

include dirs: string []

smoothdeveloper · 2020-01-26T21:53:31Z

src/fsharp/DependencyManager.Integration.fs

+        yield AppDomain.CurrentDomain.BaseDirectory
+    ])
+
+let enumerateDependencyManagerAssemblies compilerTools m =


Is this going to conform to the RFC https://github.com/fsharp/fslang-design/blob/master/tooling/FST-1027-fsi-references.md#handler-resolution or does the RFC need to be adjusted?

KevinRansom · 2020-01-27T22:29:46Z

@smoothdeveloper, Excellent, now I can see it, thanks

cartermp previously requested changes Oct 31, 2018

View reviewed changes

src/buildfromsource/FSharp.Compiler.Private/FSComp.resx Outdated Show resolved Hide resolved

cartermp mentioned this pull request Nov 1, 2018

Tricky to reference .NET Standard 2.0 components from F# Interactive #3309

Closed

cartermp mentioned this pull request Nov 1, 2018

[WIP]Adding paket package resolution to fsi and VF# editor #2483

Closed

cartermp reviewed Nov 2, 2018

View reviewed changes

enricosada reviewed Nov 2, 2018

View reviewed changes

src/fsharp/FSharp.DependencyManager/FSharp.DependencyManager.Utilities.fs Outdated Show resolved Hide resolved

brettfo approved these changes Aug 1, 2019

View reviewed changes

KevinRansom changed the base branch from release/fsharp48 to release/fsharp5 August 9, 2019 17:17

rfrerebe mentioned this pull request Sep 3, 2019

Handle fsx file with dotnet core 3 fable-compiler/Fable#1892

Merged

KevinRansom and others added 8 commits September 9, 2019 12:05

CompilerTools switch initial implementation

1403751

Dependency Manager

c47c367

Dependency Manager

5801a62

back to almost working -- needs a ton of work

e569606

Use target framework from host

fe6c1a0

Support Language Version switch

0bfe4ca

remove extra fsi

78bfdac

Fix build

0fcfe15

KevinRansom force-pushed the fsi-packetmanagement branch from f4afd65 to 0fcfe15 Compare September 9, 2019 19:45

KevinRansom and others added 4 commits September 9, 2019 15:38

Redo solution

c3763d3

remove pdb from nuget package.

5167f64

Merge branch 'release/fsharp5' of https://github.com/dotnet/fsharp in…

076153e

…to fsi-packetmanagement

Fix for missing deps.json

d48fb59

brettfo approved these changes Sep 11, 2019

View reviewed changes

assembly name

05329e3

KevinRansom merged commit c50a086 into dotnet:release/fsharp5 Sep 12, 2019

zyzhu mentioned this pull request Sep 18, 2019

Dll hardcoded dependencies on Newtonsoft.Json Google.DataTable.Net.Wrapper fslaborg/XPlot#87

Closed

David-Engel mentioned this pull request Dec 3, 2019

Cannot use with F# scripts dotnet/SqlClient#292

Closed

smoothdeveloper reviewed Jan 16, 2020

View reviewed changes

smoothdeveloper reviewed Jan 26, 2020

View reviewed changes

Package Management for FSI #5850

Package Management for FSI #5850

Conversation

KevinRansom commented Oct 31, 2018

Notes:

KevinRansom commented Oct 31, 2018

cartermp commented Oct 31, 2018

dsyme commented Oct 31, 2018

forki commented Nov 1, 2018 via email

dsyme commented Nov 1, 2018

forki commented Nov 1, 2018

cartermp commented Nov 1, 2018

forki commented Nov 1, 2018 via email

KevinRansom commented Nov 1, 2018

cartermp commented Nov 1, 2018

cartermp commented Nov 1, 2018 • edited Loading

KevinRansom commented Nov 1, 2018

cartermp commented Nov 1, 2018

KevinRansom commented Nov 1, 2018

cartermp commented Nov 1, 2018

KevinRansom commented Nov 1, 2018

KevinRansom commented Nov 1, 2018

Choose a reason for hiding this comment

enricosada Nov 2, 2018 • edited Loading

Choose a reason for hiding this comment

enricosada commented Nov 2, 2018

KevinRansom commented Nov 2, 2018

cartermp commented Nov 2, 2018 • edited Loading

forki commented Nov 2, 2018 via email

KevinRansom commented Nov 2, 2018 via email

matthid commented Nov 2, 2018

KevinRansom commented Nov 2, 2018

goswinr commented Oct 25, 2019 • edited Loading

realvictorprm commented Oct 25, 2019

goswinr commented Oct 25, 2019

KevinRansom commented Oct 25, 2019

KevinRansom commented Oct 25, 2019

goswinr commented Oct 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smoothdeveloper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KevinRansom commented Jan 27, 2020

cartermp commented Nov 1, 2018 •

edited

Loading

enricosada Nov 2, 2018 •

edited

Loading

cartermp commented Nov 2, 2018 •

edited

Loading

goswinr commented Oct 25, 2019 •

edited

Loading