musescore: 4.0.2 -> 4.1.0 #243298

I'll wait a few days for help from Darwin users, and if none will come, I'll do merge the previous dmg based expression by @yurrriq into default.nix.

reckenrode · 2023-07-15T15:13:18Z

I’m replying here as well to document (from https://discourse.nixos.org/t/need-help-from-darwin-users-syntax-errors-in-library-frameworks-foundation-framework-headers/30467/2). The issue is portaudio is propagating frameworks from the 10.12 SDK. Mixing frameworks from different SDK versions doesn’t work. It results in errors because the headers aren’t compatible. You need to override portaudio to use the 11.0 SDK.

let
  # portaudio propagates Darwin frameworks. Rebuild it using the 11.0 stdenv from Qt and the 11.0 SDK frameworks.
  portaudio' = if !stdenv.isDarwin
    then portaudio
    else portaudio.override {
      inherit stdenv;
      inherit (darwin.apple_sdk_11_0.frameworks) AudioUnit AudioToolbox CoreAudio CoreServices Carbon;
    };
in

I also want to note this derivation doesn’t build with sandbox enabled. This is a Qt issue. qtbase should be propagating its sandbox profile, but it doesn’t actually seem to work. This is the current workaround I’ve used in a couple of other packages:

# HACK `propagatedSandboxProfile` does not appear to actually propagate the sandbox profile from `qtbase`
sandboxProfile = toString qtbase.__propagatedSandboxProfile or null;

doronbehar · 2023-07-15T16:09:46Z

@ofborg build musescore

Thanks for the detailed answer @reckenrode ! Hopefully this will work now. I wonder why the default darwin frameworks are not 11_0 throughout all Nixpkgs... Perhaps, portaudio and portmidi should not propagate these deps?

doronbehar · 2023-07-15T16:12:18Z

Still very similar syntax errors...

reckenrode · 2023-07-15T16:49:57Z

I wonder why the default darwin frameworks are not 11_0 throughout all Nixpkgs

The x86_64-darwin SDK hasn’t been bumped in a while. @toonn is working on a bump to 10.13. As for why, I think there is a desire to use a source-based SDK when possible as well as one to support as many users as possible.

I don’t see needing to support multiple SDKs changing. Some software requires a newer one or should be built with a newer one (e.g., Blender) then back deployed (e.g., MoltenVK). Some could be built with a newer one and work okay, but presumably a lot of software does static feature detection instead of dynamic, so we have to build with as old as possible to maximize support. There are definitely some ergonomic and composability issues with the current approach (SDK-specific callPackages). I opened #242666 to discuss the future of SDKs on Darwin.

doronbehar · 2023-07-15T17:01:31Z

I opened #242666 to discuss the future of SDKs on Darwin.

Thanks for the link! Do you have any idea what can be done with the current compilation issue? It's availalbe here.

reckenrode · 2023-07-15T18:05:53Z

Thanks for the link! Do you have any idea what can be done with the current compilation issue? It's availalbe here.

portmidi doesn’t propagate its build inputs, so you don’t have to rebuild it. It also appears I was wrong about the stdenv. I thought Qt would provide an 11.0 SDK one, but it’s apparently still the default 10.12 SDK stdenv. Using the 11.0 SDK stdenv directly allows it to build for me. I apologize for not doing more testing (aside from confirming the expression for portaudio). Here’s a diff with the changes I made to get it to build for x86_64-darwin.

diff --git a/pkgs/applications/audio/musescore/default.nix b/pkgs/applications/audio/musescore/default.nix
index 07a9e36a0ab..da0c8110a66 100644
--- a/pkgs/applications/audio/musescore/default.nix
+++ b/pkgs/applications/audio/musescore/default.nix
@@ -31,10 +31,11 @@
 }:
 
 let
+  stdenv' = if stdenv.isDarwin then darwin.apple_sdk_11_0.stdenv else stdenv;
   # portaudio and portmidi propagate Darwin frameworks. Rebuild them using the
   # 11.0 stdenv from Qt and the 11.0 SDK frameworks.
   portaudio' = if stdenv.isDarwin then portaudio.override {
-      inherit stdenv;
+      stdenv = stdenv';
       inherit (darwin.apple_sdk_11_0.frameworks)
         AudioUnit
         AudioToolbox
@@ -43,17 +44,7 @@ let
         Carbon
       ;
     } else portaudio;
-  portmidi' = if stdenv.isDarwin then portmidi.override {
-      inherit stdenv;
-      inherit (darwin.apple_sdk_11_0.frameworks)
-        Carbon
-        CoreAudio
-        CoreFoundation
-        CoreMIDI
-        CoreServices
-      ;
-    } else portmidi;
-in stdenv.mkDerivation rec {
+in stdenv'.mkDerivation rec {
   pname = "musescore";
   version = "4.1.0";
 
@@ -101,7 +92,7 @@ in stdenv.mkDerivation rec {
     libsndfile
     libvorbis
     portaudio'
-    portmidi'
+    portmidi
     flac
     qtbase
     qtdeclarative

Unfortunately, it fails during install phase. At least it’s not SDK related ….

CMake Error at src/app/cmake_install.cmake:87 (file):
  file INSTALL cannot find
  "/private/tmp/nix-build-qtbase-5.15.9.drv-0/qtbase-a196623/$(out)/$(qtQmlPrefix)":
  No such file or directory.
Call Stack (most recent call first):
  src/cmake_install.cmake:42 (include)
  cmake_install.cmake:57 (include)


FAILED: CMakeFiles/install.util
cd /tmp/nix-build-musescore-4.1.0.drv-0/source/build && /nix/store/q6cc8rhv9v8ji23xw3d8rp2ns0cclzc1-cmake-3.26.4/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.

drupol · 2023-07-15T19:56:43Z

pkgs/applications/audio/musescore/default.nix

@@ -54,6 +54,9 @@ in stdenv'.mkDerivation rec {
    rev = "v${version}";
    sha256 = "sha256-CqW1f0VsF2lW79L3FY2ev+6FoHLbYOJ9LWHeBlWegeU=";
  };
+  patches = [
+    ./darwin-fix-install.patch


Can you please add a comment on why this patch is required?

Can you please add a comment on why this patch is required?

I will, once I see that it works.

reckenrode · 2023-07-15T23:42:19Z

I was able to get it to build, but I ran into a few issues.

It hardcodes x86_64 by default. I had to apply a fix in postPatch.
The app bundle should be installed to $out/Applications.
The wrapper args need tweaked to distinguish between Darwin and Linux.
It was failing to open due to missing QtQuick.Dialogs. I had to add qtquickcontrols as a buildInput.
It appears there is some third-party dep stuff installed to $out/lib and $out/include. Is that needed?

The following diff resolves these issues (except the last one). Testing was confirming it launched and that I could place notes on things in a document.

diff --git a/pkgs/applications/audio/musescore/default.nix b/pkgs/applications/audio/musescore/default.nix
index 56185fccf0d..4b344516c11 100644
--- a/pkgs/applications/audio/musescore/default.nix
+++ b/pkgs/applications/audio/musescore/default.nix
@@ -20,6 +20,7 @@
 , qtdeclarative
 , qtgraphicaleffects
 , flac
+, qtquickcontrols
 , qtquickcontrols2
 , qtscript
 , qtsvg
@@ -70,7 +71,8 @@ in stdenv'.mkDerivation rec {
 
   qtWrapperArgs = [
     # MuseScore JACK backend loads libjack at runtime.
-    "--prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath [ libjack2 ]}"
+    "--prefix ${lib.optionalString stdenv.isDarwin "DY"}LD_LIBRARY_PATH : ${lib.makeLibraryPath [ libjack2 ]}"
+  ] ++ lib.optionals (!stdenv.isDarwin) [
     # There are some issues with using the wayland backend, see:
     # https://musescore.org/en/node/321936
     "--set-default QT_QPA_PLATFORM xcb"
@@ -100,6 +102,7 @@ in stdenv'.mkDerivation rec {
     qtbase
     qtdeclarative
     qtgraphicaleffects
+    qtquickcontrols
     qtquickcontrols2
     qtscript
     qtsvg
@@ -110,6 +113,16 @@ in stdenv'.mkDerivation rec {
     alsa-lib
   ];
 
+  postPatch = lib.optionalString stdenv.isDarwin ''
+    substituteInPlace build/cmake/SetupBuildEnvironment.cmake \
+      --replace 'set(CMAKE_OSX_ARCHITECTURES x86_64)' 'set(CMAKE_OSX_ARCHITECTURES ${stdenv.hostPlatform.darwinArch})'
+  '';
+
+  postInstall = lib.optionalString stdenv.isDarwin ''
+    mkdir -p "$out/Applications"
+    mv "$out/mscore.app" "$out/Applications/mscore.app"
+  '';
+
   passthru.tests = nixosTests.musescore;
 
   meta = with lib; {

reckenrode · 2023-07-15T23:51:04Z

I did a search for a file I could open. Playback works too (though it crashes if you quit during playback).

doronbehar · 2023-07-16T09:30:06Z

Thanks again for the help @reckenrode . Regarding the CMAKE_OSX_ARCHITECTURES x86_64 thing: So you tested this build on an m1 aarch64 darwin machine? I wonder whether we can / should contribute this fix, and my patch upstream.

Also, regarding the postInstall, Do you get eventually also a $out/bin/mscore binary? Is $out/Applications/mscore.App needed in addition to $out/bin/mscore?

doronbehar · 2023-07-16T09:31:56Z

Oh and BTW I was wondering what kind of error do you get when sandboxProfile is not set to that qtbase.__propagatedSandboxProfile...

doronbehar · 2023-07-16T10:35:37Z

pkgs/applications/audio/musescore/default.nix

+  # Upstream assumes the user has x86_64, and not aarch64
+  postPatch = lib.optionalString stdenv.isDarwin ''
+    substituteInPlace build/cmake/SetupBuildEnvironment.cmake \
+      --replace 'set(CMAKE_OSX_ARCHITECTURES x86_64)' 'set(CMAKE_OSX_ARCHITECTURES ${stdenv.hostPlatform.darwinArch})'


I wonder whether this can be set with a cmakeFlag (question for @reckenrode ).

There is BUILD_MACOS_APPLE_SILICON, which can be set to ON. It works by clearing out CMAKE_OSX_ARCHITECTURES, which (as I understand it) means to build for the native host only. That should also work normally, but I’m not sure what it would do in a cross build.

I tried cross-building musescore (because x86_64-darwin to aarch64-darwin should work), but it appears waf needs an emulator for an aarch64-darwin cross-build, and there isn’t one available.

There is BUILD_MACOS_APPLE_SILICON, which can be set to ON. It works by clearing out CMAKE_OSX_ARCHITECTURES, which (as I understand it) means to build for the native host only. That should also work normally, but I’m not sure what it would do in a cross build.

Cross compilation is a very high ideal for this package IMO. I removed that substituteInPlace in favor of the mentioned cmake flag which I think will survive updates better, especially if upstream will rewrite/rename their cmake files.

reckenrode · 2023-07-16T16:54:29Z

Thanks again for the help @reckenrode . Regarding the CMAKE_OSX_ARCHITECTURES x86_64 thing: So you tested this build on an m1 aarch64 darwin machine?

I tested on an M1 Max MBP.

I wonder whether we can / should contribute this fix, and my patch upstream.

I assume such a patch would need to make copying the QML files conditional (e.g., on MUE_USE_SYSTEM_QML) because upstream probably wants to continue to embed them along with the Qt frameworks.

Also, regarding the postInstall, Do you get eventually also a $out/bin/mscore binary? Is $out/Applications/mscore.App needed in addition to $out/bin/mscore?

There’s no $out/bin/mscore. It only creates the app bundle. If you also want $out/bin/mscore, you’ll need to symlink $out/applications/mscore.app/Contents/MacOS/mscore to $out/bin/mscore.

I also wanted to note there are the following things in $out. These don’t look related or required by musescore. If not, can they removed from $out?

result/include:
gmock/  gtest/  kddockwidgets/  opus/

result/lib:
cmake/  libgmock.a  libgmock_main.a  libgtest.a  libgtest_main.a  libkddockwidgets.a  libopus.a  pkgconfig/

reckenrode · 2023-07-16T19:36:23Z

Oh and BTW I was wondering what kind of error do you get when sandboxProfile is not set to that qtbase.__propagatedSandboxProfile...

One of the Qt applications using during the build process will crash during when the sandbox is enabled. See #234122 for details.

In theory, the profile should be propagated to anything using qtbase. It doesn’t seem to work though (see #237458).

doronbehar · 2023-07-17T12:44:12Z

I also wanted to note there are the following things in $out. These don’t look related or required by musescore. If not, can they removed from $out?

Defninitly, these are not darwin specific. I removed them.

There’s no $out/bin/mscore. It only creates the app bundle. If you also want $out/bin/mscore, you’ll need to symlink $out/applications/mscore.app/Contents/MacOS/mscore to $out/bin/mscore.

OK good, I created that symlink.

I assume such a patch would need to make copying the QML files conditional (e.g., on MUE_USE_SYSTEM_QML) because upstream probably wants to continue to embed them along with the Qt frameworks.

Yes I agree. I opened an issue there: musescore/MuseScore#18665 .

Other then that, I squashed all the darwin related changes to 1 commit, and I'm pretty sure it should work. Thanks once more @reckenrode for all the help.

pkgs/applications/audio/musescore/default.nix

Co-authored-by: Randy Eckenrode <randy@largeandhighquality.com>

doronbehar · 2023-07-18T07:10:13Z

CI is fully green right now. I think this is ready to go. Any other objections / comments?

drupol · 2023-07-18T07:22:12Z

There's one missing test and it's ready to merge after that. Nice team work in this PR !

drupol · 2023-07-18T15:37:34Z

Ok enough waiting.

yurrriq · 2023-07-19T18:33:08Z

cc @yurrriq per #23204 .

@doronbehar et al. thanks for this PR! Six years ago 😮 I just did a naive port of the Homebrew... recipe? to get a quick and dirty (and binary) expression for darwin. I'm glad to see this finally getting fixed.

doronbehar force-pushed the pkg/musescore branch from cfaf838 to e48d2c2 Compare July 13, 2023 15:41

doronbehar mentioned this pull request Jul 13, 2023

Update request: MuseScore 4.0.2 → 4.1 #243109

Closed

1 task

doronbehar force-pushed the pkg/musescore branch from e48d2c2 to c5c5ebe Compare July 13, 2023 15:53

turion reviewed Jul 13, 2023

View reviewed changes

pkgs/applications/audio/musescore/default.nix Show resolved Hide resolved

musescore: 4.0.2 -> 4.1.0

47919de

Diff: musescore/MuseScore@v4.0.2...v4.1.0

doronbehar force-pushed the pkg/musescore branch from c5c5ebe to 47919de Compare July 13, 2023 16:24

ofborg bot requested a review from turion July 13, 2023 17:21

ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Jul 13, 2023

SuperSandro2000 reviewed Jul 13, 2023

View reviewed changes

pkgs/applications/audio/musescore/darwin.nix Outdated Show resolved Hide resolved

doronbehar added 2 commits July 15, 2023 11:30

musescore: format with nixpkgs-fmt

85b7563

musescore: Switch to wrapQtAppsHook and stdenv.mkDerivation

4416f5b

The usage of libsForQt5's mkDerivation is deprecated.

doronbehar force-pushed the pkg/musescore branch from 42577fc to 5514d0c Compare July 15, 2023 08:31

ofborg bot added the 8.has: clean-up label Jul 15, 2023

doronbehar force-pushed the pkg/musescore branch 2 times, most recently from 1f4dd1b to edadd2d Compare July 15, 2023 13:32

drupol reviewed Jul 15, 2023

View reviewed changes

doronbehar force-pushed the pkg/musescore branch from 5fcf3e3 to bc14b92 Compare July 16, 2023 10:34

doronbehar commented Jul 16, 2023

View reviewed changes

doronbehar added 2 commits July 17, 2023 15:27

musescore: Remove unneeded qt deps from buildInputs

3b32c6f

musescore: remove unneeded bundled libraries and headers from $out

014929a

doronbehar force-pushed the pkg/musescore branch from bc14b92 to 12b4ed6 Compare July 17, 2023 12:29

doronbehar mentioned this pull request Jul 17, 2023

Darwin Build from source: Don't always copy qml files from qtbase musescore/MuseScore#18665

Closed

doronbehar force-pushed the pkg/musescore branch from 12b4ed6 to c9db298 Compare July 17, 2023 12:41

NixOS deleted a comment from doronbehar Jul 17, 2023

SuperSandro2000 reviewed Jul 17, 2023

View reviewed changes

pkgs/applications/audio/musescore/default.nix Show resolved Hide resolved

doronbehar force-pushed the pkg/musescore branch from c9db298 to f0a79ec Compare July 17, 2023 16:05

musescore: Fix darwin build from source

8895d55

Co-authored-by: Randy Eckenrode <randy@largeandhighquality.com>

doronbehar force-pushed the pkg/musescore branch from f0a79ec to 8895d55 Compare July 17, 2023 16:31

drupol merged commit 446b09f into NixOS:master Jul 18, 2023

turion mentioned this pull request Jul 20, 2023

Musescore 4.1.0 segfault on launch #244409

Closed

cpick mentioned this pull request Jan 24, 2024

Build failure: Wezterm (x86-64 darwin) #239384

Closed

shivaraj-bh mentioned this pull request Mar 22, 2024

grafana: fix build failure on darwin #297968

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

musescore: 4.0.2 -> 4.1.0 #243298

musescore: 4.0.2 -> 4.1.0 #243298

doronbehar commented Jul 13, 2023

colin-heffernan commented Jul 13, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023 •

edited

Loading

nixos-discourse commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

drupol Jul 15, 2023

doronbehar Jul 15, 2023

reckenrode commented Jul 15, 2023 •

edited

Loading

reckenrode commented Jul 15, 2023 •

edited

Loading

doronbehar commented Jul 16, 2023

doronbehar commented Jul 16, 2023

doronbehar Jul 16, 2023

reckenrode Jul 16, 2023

doronbehar Jul 17, 2023

reckenrode commented Jul 16, 2023

reckenrode commented Jul 16, 2023

doronbehar commented Jul 17, 2023

doronbehar commented Jul 18, 2023

drupol commented Jul 18, 2023

drupol commented Jul 18, 2023

yurrriq commented Jul 19, 2023

musescore: 4.0.2 -> 4.1.0 #243298

musescore: 4.0.2 -> 4.1.0 #243298

Conversation

doronbehar commented Jul 13, 2023

Description of changes

Things done

colin-heffernan commented Jul 13, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023 • edited Loading

nixos-discourse commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

doronbehar commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

doronbehar commented Jul 15, 2023

reckenrode commented Jul 15, 2023

drupol Jul 15, 2023

Choose a reason for hiding this comment

doronbehar Jul 15, 2023

Choose a reason for hiding this comment

reckenrode commented Jul 15, 2023 • edited Loading

reckenrode commented Jul 15, 2023 • edited Loading

doronbehar commented Jul 16, 2023

doronbehar commented Jul 16, 2023

doronbehar Jul 16, 2023

Choose a reason for hiding this comment

reckenrode Jul 16, 2023

Choose a reason for hiding this comment

doronbehar Jul 17, 2023

Choose a reason for hiding this comment

reckenrode commented Jul 16, 2023

reckenrode commented Jul 16, 2023

doronbehar commented Jul 17, 2023

doronbehar commented Jul 18, 2023

drupol commented Jul 18, 2023

drupol commented Jul 18, 2023

yurrriq commented Jul 19, 2023

doronbehar commented Jul 15, 2023 •

edited

Loading

reckenrode commented Jul 15, 2023 •

edited

Loading

reckenrode commented Jul 15, 2023 •

edited

Loading