Build mulitple projects in a single dotnet fable run to optimize build time

[DunetsNM]

Something I came up with while working with --precompiledLib switch, which is the current way to optimize build time.

Scenario

I have a project with three different Fable apps (each is a separate .fsproj file) that share a lot of code via other directly and indirectly referenced F# projects. In fact most of the code is shared (about ~700 .fs files out of 1000-1200 total for each app).

All three build and released together in a single CI pipeline. I need a separate dotnet fable to build each of the projects. By default it'll compile each app separately from the sources, rebuilding all those ~700 shared files 3 times.

To optimize build time we use precompilation as suggested in this blog post about compilation speed.

There are some downside to it though:

• it becomes app developers' job to figure out which libraries should be precompiled and reused, which adds cognitive overhead
• precompiled files are not watched
• apparently it adds complexity to compiler and is easy enough to break (as it did between Fable 3 and Fable 4 releases)

Experiment

While compiling an individual .fsproj Fable produces flat and distinct list of files and compiles them in correct order via F# compiler services.
So I did a quick experiment and created a dummy project AppsAll.fsproj that does nothing but references all of my 3 apps projects, and tried to compile it with fable (without any precompilation!) :

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
    </PropertyGroup>

    <ItemGroup>
        <ProjectReference Include="../App1/src/App1.fsproj" />
        <ProjectReference Include="../App2/src/App2.fsproj" />
		<ProjectReference Include="../App3/src/App3.fsproj" />
    </ItemGroup>
</Project>

To my excitement Fable compiled it just fine! And much faster than compiling each app project separately - because each dependency source file was compiled only once (Fable did much better job reusing code here than I did via the precompiled lib - and did it automatically)

There's one problem: the output of this dummy project is also dumped in one directory

./App1/
./App2/
./App3/
./Lib1/
./Lib2/
...
./LibN/
./fable_modules/ - nuget packages go here

where Lib1 to LibN and ./fable_modules/ is the union of all dependencies of App1 , App2 and App3

I probably can just copy the output to three separate directories and tree-shake them somehow for App 1 to 3 respectively, but even if it's easy it just feels like a hack.

Idea

It'd be nice if instead of creating the dummy .fsproj file and doing dark magic to the output, I could just pass App1.fsproj App2.fsproj and App3.fsproj to fable in a single command line and get three separate output folders, one per app - all while compiling all F# source files only once i.e. faster (while avoiding using precompiled lib dll altogether)

[MangelMaxime]

This discussion reminds me a little of #3737

There is an important difference between consuming Fable compiled file and precompiled libraries.

Fable compiled file cannot consume inlined code. This is because, in JavaScript the inlined information is not present anymore it is "inject" by the compiler at compilation if needed by the caller function.

This is why precompiled libraries generates both the dll and the json file. If I remember correctly and understood correctly the dll contains the code while the inline information is kept in the json so Fable can still apply inline functions correctly.

[ncave]

@DunetsNM

I probably can just copy the output to three separate directories and tree-shake them somehow for App 1 to 3 respectively, but even if it's easy it just feels like a hack.

To me it sounds more like building a solution file with multiple project in it. Perhaps that's what we want to support.
Not sure if FCS supports building solution files out of the box, but even if it doesn't, in theory it should be possible, if we extend the new project cracker that uses MSBuild.

But that would still be like building each project separately, so probably not what you're trying to do.

[DunetsNM]

my first thought was also a solution file. As a matter of fact I have a .sln file but it has all the dependency/lib projects too to open nicely in IDE. Thing is, I don't want to have a separate build output for each of them (it will probably be slower) but only for the app projects, but there's no way to tell "proper" projects to build from just dependency projects (at least don't know one), it's very Fable-specific

[ncave]

@DunetsNM

There's one problem: the output of this dummy project is also dumped in one directory

Have you tried moving the (dummy) "solution" project to a parent folder (to /App1, /App2 /App3), instead of a sibling folder? Not sure that helps in any way, it's probably the same.

Would using some sort of bundler as post-build for each App help?

[DunetsNM]

Would using some sort of bundler as post-build for each App help?

It probably would it's my current workaround plan, will see.

Have you tried moving the (dummy) "solution" project to a parent folder

Interesting. I'll experiment with relative dir layout but I don't think it matters much. As a matter of fact there's a plan to move our libs to a private nuget feed so they will land in fable_modules along with other packages, but further tree-shaking / packaging will be required anyway.

[DunetsNM]

Just in case, at the moment I have a mono repo of roughly this format:

./SharedLib1/
./SharedLib2/
...
./SharedLibN/
./SolutionA/LocalLibA1/
./SolutionA/AppA1/
./SolutionA/AppA2/
...
./SolutionX/LocalLibX1
./SolutionX/AppX1

each Solution is a separate product / CI pipeline.
i.e. dependencies appear either as sibling dir to app or above it. Interesting that Fable has no problems with that and coerces those shared libs higher in the hierarchy into the same output folder.

[ncave]

@DunetsNM
Yes, Fable puts all project file dependencies under the output folder (if provided).
But a tree-shaking bundler like say rollup should be able to pick only the code actually used for each App.