Use OmniSharp's embedded Mono version for integration tests.

[bjorkstromm]

No description provided.

[bjorkstromm]

@gep13 or @devlead are you able to reproduce the errors in Bitrise locally on your Macs? Not the same errors we are seeing in OmniSharp though, but it's blocking the actual integration test (i.e. can't install Bakery from locally created nupkg).

[bjorkstromm]

@DustinCampbell, as you can see we are adding integration tests for running Bakery with the same mono version's embedded with OmniSharp. Currently hitting an issue with installing the locally created nupkg, so we can't really see yet if the hangs you are experiencing on OmniSharp is due to Bakery or not. But, we'll keep you posted!

[bjorkstromm]

Something weird going on when NuGet libraries are unzipping the NuPkg.

Error: NuGet.Protocol.Core.Types.FatalProtocolException: '/Users/vagrant/git/tests/integration/packages/Cake.Bakery.0.3.0-unstable0023.nupkg' is not a valid nupkg file. Error: '/Library/Frameworks/Mono.framework/Versions/5.10.0/lib/../lib/libMonoPosixHelper.dylib'. ---> System.DllNotFoundException: /Library/Frameworks/Mono.framework/Versions/5.10.0/lib/../lib/libMonoPosixHelper.dylib
  at (wrapper managed-to-native) System.IO.Compression.DeflateStreamNative.CreateZStream(System.IO.Compression.CompressionMode,bool,System.IO.Compression.DeflateStreamNative/UnmanagedReadOrWrite,intptr)
  at System.IO.Compression.DeflateStreamNative.Create (System.IO.Stream compressedStream, System.IO.Compression.CompressionMode mode, System.Boolean gzip) [0x0004a] in <ecca2eb37e554a54afd630a35ee09715>:0 
  at System.IO.Compression.DeflateStream..ctor (System.IO.Stream compressedStream, System.IO.Compression.CompressionMode mode, System.Boolean leaveOpen, System.Boolean gzip) [0x0002d] in <ecca2eb37e554a54afd630a35ee09715>:0 
  at System.IO.Compression.DeflateStream..ctor (System.IO.Stream stream, System.IO.Compression.CompressionMode mode) [0x00000] in <ecca2eb37e554a54afd630a35ee09715>:0 
  at (wrapper remoting-invoke-with-check) System.IO.Compression.DeflateStream..ctor(System.IO.Stream,System.IO.Compression.CompressionMode)
  at System.IO.Compression.ZipArchiveEntry.GetDataDecompressor (System.IO.Stream compressedStreamToRead) [0x00017] in <080bae4fb146443298e31a573adbaaf9>:0 
  at System.IO.Compression.ZipArchiveEntry.OpenInReadMode (System.Boolean checkOpenable) [0x00028] in <080bae4fb146443298e31a573adbaaf9>:0 
  at System.IO.Compression.ZipArchiveEntry.Open () [0x00026] in <080bae4fb146443298e31a573adbaaf9>:0 
  at NuGet.Packaging.ZipArchiveExtensions.OpenFile (System.IO.Compression.ZipArchive zipArchive, System.String path) [0x00007] in <67f0eff528bd46878f4bdc3452e16850>:0 
  at NuGet.Packaging.PackageArchiveReader.GetStream (System.String path) [0x0001c] in <67f0eff528bd46878f4bdc3452e16850>:0 
  at NuGet.Packaging.PackageReaderBase.GetNuspec () [0x00007] in <67f0eff528bd46878f4bdc3452e16850>:0 
  at NuGet.Packaging.PackageReaderBase.get_NuspecReader () [0x00008] in <67f0eff528bd46878f4bdc3452e16850>:0 
  at NuGet.Protocol.LocalFolderUtility.GetPackageFromNupkg (System.IO.FileInfo nupkgFile) [0x00025] in <e4b64921d3e6473eaea83624572a8cb9>:0 
   --- End of inner exception stack trace ---
  at NuGet.Protocol.LocalFolderUtility.GetPackageFromNupkg (System.IO.FileInfo nupkgFile) [0x000bb] in <e4b64921d3e6473eaea83624572a8cb9>:0 
  at NuGet.Protocol.LocalFolderUtility+<>c.<GetPackagesFromNupkgs>b__31_0 (System.IO.FileInfo file) [0x00000] in <e4b64921d3e6473eaea83624572a8cb9>:0 
  at System.Linq.Enumerable+SelectEnumerableIterator`2[TSource,TResult].MoveNext () [0x00048] in <bb236b1d795f45a39f8b3c2d7f14b461>:0 
  at NuGet.Protocol.LocalFolderUtility+<GetPackagesV2>d__3.MoveNext () [0x000cc] in <e4b64921d3e6473eaea83624572a8cb9>:0 
  at NuGet.Protocol.LocalFolderUtility+<GetDistinctPackages>d__27.MoveNext () [0x0008e] in <e4b64921d3e6473eaea83624572a8cb9>:0 
  at NuGet.Protocol.LocalDependencyInfoResource.ResolvePackages (System.String packageId, NuGet.Frameworks.NuGetFramework projectFramework, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x0006b] in <e4b64921d3e6473eaea83624572a8cb9>:0 
  at Cake.NuGet.Install.NuGetPackageInstaller.GetNuGetVersion (Cake.Core.Packaging.PackageReference package, Cake.NuGet.Install.NuGetSourceRepositoryProvider sourceRepositoryProvider, NuGet.Frameworks.NuGetFramework targetFramework, NuGet.Common.ILogger logger) [0x0006d] in <c0e6495c03054c72bbdbbd15b0e249dd>:0 
  at Cake.NuGet.Install.NuGetPackageInstaller.GetPackageId (Cake.Core.Packaging.PackageReference package, Cake.NuGet.Install.NuGetSourceRepositoryProvider sourceRepositoryProvider, NuGet.Frameworks.NuGetFramework targetFramework, NuGet.Common.ILogger logger) [0x00000] in <c0e6495c03054c72bbdbbd15b0e249dd>:0 
  at Cake.NuGet.Install.NuGetPackageInstaller.Install (Cake.Core.Packaging.PackageReference package, Cake.Core.Packaging.PackageType type, Cake.Core.IO.DirectoryPath path) [0x00053] in <c0e6495c03054c72bbdbbd15b0e249dd>:0 
  at Cake.Core.Scripting.ScriptProcessor.InstallPackages (System.Collections.Generic.IReadOnlyCollection`1[T] modules, Cake.Core.IO.DirectoryPath installPath, Cake.Core.Packaging.PackageType packageType) [0x00091] in <0d321134251b415e880332415ed6a0d0>:0 
  at Cake.Core.Scripting.ScriptProcessor.InstallTools (System.Collections.Generic.IReadOnlyCollection`1[T] tools, Cake.Core.IO.DirectoryPath installPath) [0x0001c] in <0d321134251b415e880332415ed6a0d0>:0 
  at Cake.Core.Scripting.ScriptRunner.Run (Cake.Core.Scripting.IScriptHost host, Cake.Core.IO.FilePath scriptPath, System.Collections.Generic.IDictionary`2[TKey,TValue] arguments) [0x00125] in <0d321134251b415e880332415ed6a0d0>:0 
  at Cake.Commands.BuildCommand.Execute (Cake.CakeOptions options) [0x00026] in <7d05293a8fea4f9a8888ab9a09062744>:0 
  at Cake.CakeApplication.Run (Cake.CakeOptions options) [0x00015] in <7d05293a8fea4f9a8888ab9a09062744>:0 
  at Cake.Program.Main () [0x000d1] in <7d05293a8fea4f9a8888ab9a09062744>:0 
An error occurred when executing task 'Run-Bakery-Integration-Tests'.

[bjorkstromm]

Could not load /Users/vagrant/git/tools/Cake/Cake.Common.dll (missing System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a))

Seems like at least System.Net.Http is missing from the bundled Mono version on Mac.

[DustinCampbell]

FWIW, the exception shows that it was probing for /Library/Frameworks/Mono.framework/Versions/5.10.0/lib/../lib/libMonoPosixHelper.dylib. However, if you're seeing that it's likely that you weren't running on the embedded Mono. The embedded Mono, doesn't know about /Library/Frameworks/Mono.framework. That is, it won't is you use the run script, which sets MONO_PATH.

[DustinCampbell]

You're right that System.Net.Http is not included. OmniSharp doesn't need that to operate.

[DustinCampbell]

Let me be a bit more precise: The embedded Mono framework does not carry System.Net.Http because OmniSharp.Stdio does not need it. OmniSharp.Http does use it, so it includes it directly.

[DustinCampbell]

FWIW, here's where that's getting thrown from in NuGet: https://github.com/NuGet/NuGet.Client/blob/8144fdeca466151c1516b65dc30d429562890b86/src/NuGet.Core/NuGet.Protocol/Utility/LocalFolderUtility.cs#L1115

[DustinCampbell]

And here's where it blows up in Mono: https://github.com/mono/mono/blob/7b4dfeebc40cf8c027819b8b7bd85a4e7c87ad50/mcs/class/System/System.IO.Compression/DeflateStream.cs#L524

[DustinCampbell]

Looking at the Mono code, it appears that we should indeed include the monoPosixHelper with the embedded Mono. I'm running a couple of OSX builds for OmniSharp with Mono runtime logging and xUnit diagnostics turned on. Once I have that information, I'll sift through the logs and see if I can verify that this is true.

[DustinCampbell]

Yeah, I can see the difference in the mono logging output on Travis.

On Linux (which passes), I see this:

Mono: Assembly Ref addref Cake.Scripting.Abstractions[0x15a7d60] -> System.Core[0x15212e0]: 25
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Assembly Ref addref Cake.Scripting.Abstractions[0x15a7d60] -> System[0x152f1a0]: 32
Mono: Assembly System.Threading is a framework Facade asseembly
Mono: Assembly System.Threading is a framework Facade asseembly
Mono: Assembly Ref addref System.Threading.Tasks.Dataflow[0x1dc0e30] -> System.Threading[0x2868f90]: 8
Mono: DllImport attempting to load: '/usr/lib/../lib/libMonoPosixHelper.so'.
Mono: DllImport loaded library '/usr/lib/../lib/libMonoPosixHelper.so'.
Mono: DllImport searching in: '/usr/lib/../lib/libMonoPosixHelper.so' ('/usr/lib/../lib/libMonoPosixHelper.so').
Mono: Searching for 'CreateZStream'.
Mono: Probing 'CreateZStream'.
Mono: Found as 'CreateZStream'.
Mono: DllImport searching in: '/usr/lib/../lib/libMonoPosixHelper.so' ('/usr/lib/../lib/libMonoPosixHelper.so').
Mono: Searching for 'ReadZStream'.
Mono: Probing 'ReadZStream'.
Mono: Found as 'ReadZStream'.
Mono: DllImport searching in: '/usr/lib/../lib/libMonoPosixHelper.so' ('/usr/lib/../lib/libMonoPosixHelper.so').
Mono: Searching for 'CloseZStream'.
Mono: Probing 'CloseZStream'.
Mono: Found as 'CloseZStream'.
Mono: Assembly Ref addref Microsoft.Extensions.Caching.Memory[0x1b27f40] -> Microsoft.Extensions.Primitives[0x1b411c0]: 10
Mono: Assembly Ref addref Microsoft.Extensions.Caching.Abstractions[0x1b22030] -> Microsoft.Extensions.Primitives[0x1b411c0]: 11

On OSX, I see this:

Mono: Assembly Ref addref Cake.Scripting.Abstractions[0x7fe108444680] -> System.Core[0x7fe108435300]: 24
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Assembly Ref addref Cake.Scripting.Abstractions[0x7fe108444680] -> System[0x7fe10842e1c0]: 32

...

Mono: Assembly Ref addref Microsoft.Extensions.Caching.Memory[0x7fe1085b90e0] -> Microsoft.Extensions.Primitives[0x7fe1085bf2a0]: 10
Mono: Assembly Ref addref Microsoft.Extensions.Caching.Abstractions[0x7fe1084ad360] -> Microsoft.Extensions.Primitives[0x7fe1085bf2a0]: 11

Note that the DllImport code is missing on OSX. Also, note that it tries to search in /usr/lib/../lib/, even though the embedded mono has set MONO_PATH. So, it looks like we'll need to pull libMonoPosixHelper with embedded Mono. I'll look at doing that tomorrow.

[DustinCampbell]

OK. I've got a mono package that includes that particular native dll. However, there's some weirdness with how Cake.Bakery.exe is run. Instead of running with the OmniSharp mono run script, it is run directly on the embedded Mono. However, in that case, none of the extra arguments get passed. That's a bit concerning. I'm taking a look at whether I can fix that by settingMONO_ENV_OPTIONS in the run script.

[DustinCampbell]

It looks like MONO_ENV_OPTIONS will do the trick.

[DustinCampbell]

Everything is working ok with the new Mono package. Here's the PR that I submitted for OmniSharp: OmniSharp/omnisharp-roslyn#1137.

[bjorkstromm]

Thanks @DustinCampbell! I will update this PR to use the new embedded mono bits.

[DustinCampbell]

Success!