Officially support building F* from source with F# 4.1 #1358
Comments
|
@ranweiler Many thanks for the detailed report. Some quick answers:
I guess you're aware that Step 1 from INSTALL.md (building via F#) is not required, you can also bootstrap F* completely in OCaml, using an old F* version extracted to OCaml (Steps 2-3). |
|
@catalin-hritcu Thanks! I did notice that Step 1 can be skipped— I just figured I'd use myself as a test case for building from scratch on Linux.
|
In case it helps narrow this down, here is a what I get on a clean F# build on a clean repo:
|
Finally managed to narrow this down: on all my machines But on the CI machine it causes only the 2nd error, but not the 1st "assertion failed" pointing at prims.fst. |
|
Thanks to @mtzguido we narrowed 3 to the |
|
Thanks for that build output on (2), @catalin-hritcu! Once difference that I see immediately is that your invocation of To get Showing the versions I'm working with: Then, when i invoke I notice in your build output you have P.S. nice investigating on (3)! If other folks are relying on CI to run tests, that would explain why this hasn't been caught yet. |
|
Note: when I try to build with the newer The first error is |
|
@ranweiler Did some testing and it seems this works with msbuild version 14.1.x-15.2.x. With 15.4 things failed catastrophically for me too: https://gist.github.com/catalin-hritcu/0d69560709e4916c9c31e380ad699449 |
|
Debugging further, I'm seeing that when defining On my system, However, this path doesn't exist on my system. Instead, the desired file can be found in I'm not yet sure about the |
|
@catalin-hritcu oh wow, so your build failures are different than mine, even. It's tough, because on Linux, it seems like there are many, many ways a user could have valid installs of the various tools. I think it would be nice if a user could just install the latest upstream |
|
Okay, adding to the above comment: it actually seems like MSBuild 15.4 support is important and inevitable, since these tools seems to be moving somewhat fast, and 15.4 is the MSBuild version that comes with the latest stable, non-LTS release of https://github.com/Microsoft/dotnet, as well as Visual Studio 2017 (see here). We really probably don't want the F* build to fail for the latest version of e.g. Visual Studio 2017 / the latest release of the .NET SDK. Unless someone else "owns" the build code and wants to do it themselves, I can take a stab at this and open a PR, though it will be important to have others review and test it carefully, as I'm new to this whole toolchain. |
|
Regarding the current state of F* with F# - F# build of F*It should be possible to build F* with F#, on both linux and windows. I personally can't merge anything but I'm sure a PR would be welcome if you want to fix building with F# 4.1, @ranweiler. The F# build uses bigint arithmetic now, like the OCaml build. As far as I'm aware there should not be any issues of unsoundness in F# but not OCaml builds. The F# build is behind in some minor areas, like unicode support. Extracted F# code#1096 is a major roadblock for getting extracted F# code to work. According to @nikswamy it may be possible to remove unused type parameters when extracting, but non-trivial. A patch to the F# compiler that would fix #1096 has been approved-in-principle, and I'm working on this right now. Because of #1096, it's not possible to realise various important parts of ulib in F# - even Prims and Pervasives, IIRC. As a result it is not possible to build ulib when extracting to F#. Your options at the moment are to either use OCaml, or to use a modified F# compiler that allows for unused type parameters when compiling extracted F# code. There are far more OCaml realised modules for ulib than there are in F#. I think most F# realisations are out of date, as well. Once #1096 is fixed, it's worth taking a look at these and bringing them up-to-date with respect to the OCaml realisations. Extracted F# code is ugly compared to extracted OCaml. It would be worth working on integrating an F# formatter like Fantomas in order to improve this slightly. Still, Fantomas is not a full solution to this and some additional work is required if we want to have extracted F# looking as good as extracted OCaml. |
|
@A-Manning thanks for the rundown of the current state of F#/F*, having that context is very helpful! For now, I'll open a PR to support the latest MSBuild release. Since @catalin-hritcu has had success with an older range of MSBuild versions, my sense is that my my PR should be enough to close this issue out, unless we want to fold the CI/local test discrepancy into this issue. However, since that discrepancy also exists on Windows, I'd suggest we make that it's own issue and address it separately. |
|
We're in the process of migrating to VS2017 and F# 4.1 and I have a new branch about that: It currently fails Windows CI though: For me this branch works on both Linux and Windows, and for what it's worth it also works on the Linux Nightly machine (although that uses xbuild which is a different beast) |
|
Lines 112 to 115 in 8612134 You need to change the references to FSharp.Core.4.2.3 to FSharp.Core.4.3.4 in your .fsproj files.
|
@A-Manning I'm not an expert on any of this but I managed to get a green on the Windows CI just by reapplying most of your fsproj changes from PR #1404. If you have further improvements now would be a good time for another pull request against my @ranweiler You also still owe us one? Anything left of that? :) |
|
@catalin-hritcu I'm traveling at the moment, but I'll catch up on this when I get back next week! |
|
@cpitclaudel Is reporting still seeing errors on the @A-Manning @ranweiler Can any of you reproduce this? (I can't.) |
|
Here: |
|
@cpitclaudel can you paste the output of |
|
|
I have the same mono and fsharpc versions, so it's probably not that. |
|
Maybe I'm doing something wrong? Hmm. |
|
I've upgraded my msbuild to 15.4.0.0 and it still works for me: I'm on Arch Linux, which is a bit different from Ubuntu, but can't think of other differences. |
|
I just set up a clean VM to experiment. I used Ubuntu 16.04, got mono from http://www.mono-project.com/download/stable/#download-lin-ubuntu, and ran this: sudo apt-get install mono-complete fsharp
git clone --depth 1 https://github.com/FStarLang/FStar.git -b catalin_fsharp-4.1-upgrade
make -j4 -C FStar/src/The build goes through, but with warnings: …and then F* isn't usable: Here's some more version info in that VM |
|
@cpitclaudel Still can't reproduce, but this seems similar to point "2. Target F# Core version" of @ranweiler's original report above. Could you try to see if a similar workaround helps for you too? Also @A-Manning's comment from a couple of weeks ago seems in that direction: |
|
Same warnings and issue after changing to 4.3.4. Which makes sense, because the package doesn't exist on disk either: I do have some version of FSharp in system folders, though: |
|
I can't reproduce this. You can try removing |
Do you mean that you can't reproduce this on your own machine, or in a clean VM? |
Should I install that, then? Our installation instructions and our nuget configuration don't say cause FSharp.Core to be installed from nuget, AFAICT. |
|
@cpitclaudel Could you try building https://github.com/A-Manning/FStar/tree/amanning_fsharp-4.1-upgrade? |
|
@nikswamy does that issue should be closed, as building F* from F# source is no longer supported? |
|
thanks. This is no longer supported. cf #2512 |
Summary
It seems like this is mostly working already. There is some discussion of this in the comments of #878, but it seemed worth breaking this out into its own issue. Presumably it also needs systematic assessment on all supported operating systems.
This did not work for me out of the box. I'd like to share what I needed to do to build F* from scratch, and hopefully we can figure out if (a) my workarounds were ultimately unnecessary, and the docs can simply be updated to claim F# 4.1 support, or (b) if the build configuration should be tweaked.
In short: most things work, but someone with more context should glance at the minor issues I came across.
There were 3 problems I ran into.
FSharp.CoreI'll address each of these below. I may refer to the numbered build-from-source steps in INSTALL.md. Note that I only ran into issues with Step 1 of those instructions.
Platform and versions
This was building the latest
master(b8ea92f).I'm running x86-64 Void Linux (kernel 4.14.5), using Mono with no prior F# or F* installs (i.e. no DLLs hanging around from previous F# versions). Specifically, I built F* with:
1. Executable format assumptions
This a somewhat trivial documentation issue. If a Linux user has Mono installed, but the CLR binary format has not been registered as an executable format, the default F* instructions for Step 1 to test the correctness of the executable do not work verbatim (unsurprisingly!). On Linux, to run a CLR executable, one can either generally use shim scripts that call e.g.
mono fstar.exe, or tell the kernel about the CLR format, as described here. For building F* from source, I chose the latter.I believe this is done by default when installing
mono-completeon Debian (at least if theApt::Install-Recommendsconfig value is set to something truthy), but I haven't tested it. On Void, this requires explicit opting-in per the instructions I just linked to. Due to the non-portability of this solution, the F* team may want to punt this to users to address as they see fit, which is valid. If so, it would be helpful to comment on this in the build docs. Otherwise, it might be nice to add shim scripts/test symlinks for Step 1 builds, which always do the right thing. I'd personally suggest the former for now.2. Target F# Core version
This issue is more substantive, and a place where I lack context.
Throughout the project, there is a configuration value named
TargetFSharpCoreVersion, set to4.4.0.0. If I build F* from sources usingmake -C src, then I observe the following warnings:This led me to expect some sort of runtime failure. Indeed, if I try to run the freshly-built CLR executable to check it, I observe.
Note
Could not load file or assembly 'FSharp.Core, Version=4.0.0.0.To fix this, I used a very crude solution. If I run
grep -F '4.4.0.0' -R ., I can find various occurrences of this version string inTargetFSharpCoreVersionXML configuration values. I did a find/replace to change every such occurrence of4.4.0.0with4.4.1.0, per this guide. This build succeeded, and I was able to run thefstar.exe --version, as well as various sub-projects in theexamplesdirectory.There may be a much better solution. For example:
apt-getaway.TargetFSharpCoreVersionneed to be updated, or maybe it is enough to update thebindingRedirectvalue I see in some config files (which I in fact never edited).My background is Unix, not Windows/the CLR, so I very much defer to others here. I assume that my heavy-handed first guess of a solution is the wrong one.
3. Successfully running all example benchmarks
I was able to run most projects in
examples/, as I mentioned, as well as perform OCaml code extraction and rebuild the F* compiler from freshly-extracted OCaml (steps 2 and 3 of INSTALL.md).However, one of the microbenchmark make targets failed (
all-neg). The expectation forNEGTESTSwas 39, but I locally produced 40 failures. I don't know if this is related to my experimentation with the build config, a CLR/CLR lib issue, a quirk of the Z3 version I'm using, a real regression inmaster, or a fluke of some other sort. If needed, I can open a new issue for it with more detail. If it is a dubious or flexible expectation in the first place, that would be helpful to know (and perhaps document).Thanks!
The text was updated successfully, but these errors were encountered: