Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl: add patch for curling directories #5

Merged
merged 1 commit into from
Apr 3, 2021

Conversation

lukegb
Copy link

@lukegb lukegb commented Apr 3, 2021

nix relies on using curl to fetch directories using the file scheme,
which got broken by the most recent curl update in
curl/curl@1e5cec3
which no longer provides a content-length for directories, but in so
doing causes any fetch of a directory to return "Couldn't resume
download (36)" (which is easily testable using the CLI)

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

nix relies on using curl to fetch directories using the file scheme,
which got broken by the most recent curl update in
curl/curl@1e5cec3
- which no longer provides a content-length for directories, but in so
doing causes *any* fetch of a directory to return "Couldn't resume
download (36)" (which is easily testable using the CLI)
@lukegb lukegb mentioned this pull request Apr 3, 2021
10 tasks
@mweinelt mweinelt merged this pull request into mweinelt:curl Apr 3, 2021
@mweinelt
Copy link
Owner

mweinelt commented Apr 3, 2021

Thanks, let's give this a try in eval.

mweinelt pushed a commit that referenced this pull request May 12, 2022
A recent change to xdg-shell-client-protocol causes a build failure.
This was reported upstream as GH #5.  User @AlexBMJ posted a two-line
fix in that issue, but the developer has not yet responded:

progandy/wdomirror#5 (comment)
mweinelt pushed a commit that referenced this pull request May 12, 2022
wdomirror: fix build failure by applying patch from GH #5
@lukegb lukegb deleted the curl-update-fixes branch December 5, 2022 01:21
mweinelt pushed a commit that referenced this pull request Jan 2, 2023
Without this change it segfaults when trying to play any media:

  $ jellyfinmediaplayer
  Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
  libpng warning: iCCP: known incorrect sRGB profile
  Logging to /home/bf/.local/share/jellyfinmediaplayer/logs/jellyfinmediaplayer.log
  Cannot load libcuda.so.1
  Segmentation fault (core dumped)

The backtrace shows pipewire being at fault:

  $ coredumpctl debug
  [...]
  Program terminated with signal SIGSEGV, Segmentation fault.
  #0  0x00007f711428c9bb in core_event_demarshal_remove_id () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  [Current thread is 1 (Thread 0x7f6ffdc87640 (LWP 1360949))]
  (gdb) bt
  #0  0x00007f711428c9bb in core_event_demarshal_remove_id () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #1  0x00007f711428886c in process_remote () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #2  0x00007f7114288e68 in on_remote_data () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #3  0x00007f7114310efe in loop_iterate () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/spa-0.2/support/libspa-support.so
  #4  0x00007f71266fe7f2 in do_loop () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/libpipewire-0.3.so.0
  #5  0x00007f7128b08e86 in start_thread () from /nix/store/ayfr5l52xkqqjn3n4h9jfacgnchz1z7s-glibc-2.35-224/lib/libc.so.6
  #6  0x00007f7128b8fce0 in clone3 () from /nix/store/ayfr5l52xkqqjn3n4h9jfacgnchz1z7s-glibc-2.35-224/lib/libc.so.6
  (gdb)

Standalone mpv doesn't segfault (when directly playing the underlying
media files). I don't know why.

Fixes: b97cda7 ("mpv-unwrapped: 0.34.1 -> 0.35.0")

Fixes NixOS#205141

Ref jellyfin/jellyfin-media-player#341
mweinelt pushed a commit that referenced this pull request Jan 11, 2023
Without this change it segfaults when trying to play any media:

  $ jellyfinmediaplayer
  Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
  libpng warning: iCCP: known incorrect sRGB profile
  Logging to /home/bf/.local/share/jellyfinmediaplayer/logs/jellyfinmediaplayer.log
  Cannot load libcuda.so.1
  Segmentation fault (core dumped)

The backtrace shows pipewire being at fault:

  $ coredumpctl debug
  [...]
  Program terminated with signal SIGSEGV, Segmentation fault.
  #0  0x00007f711428c9bb in core_event_demarshal_remove_id () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  [Current thread is 1 (Thread 0x7f6ffdc87640 (LWP 1360949))]
  (gdb) bt
  #0  0x00007f711428c9bb in core_event_demarshal_remove_id () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #1  0x00007f711428886c in process_remote () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #2  0x00007f7114288e68 in on_remote_data () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/pipewire-0.3/libpipewire-module-protocol-native.so
  #3  0x00007f7114310efe in loop_iterate () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/spa-0.2/support/libspa-support.so
  #4  0x00007f71266fe7f2 in do_loop () from /nix/store/nhffrd7f15dhfbkwzgayq7hhzmdvdy19-pipewire-0.3.63-lib/lib/libpipewire-0.3.so.0
  #5  0x00007f7128b08e86 in start_thread () from /nix/store/ayfr5l52xkqqjn3n4h9jfacgnchz1z7s-glibc-2.35-224/lib/libc.so.6
  #6  0x00007f7128b8fce0 in clone3 () from /nix/store/ayfr5l52xkqqjn3n4h9jfacgnchz1z7s-glibc-2.35-224/lib/libc.so.6
  (gdb)

Standalone mpv doesn't segfault (when directly playing the underlying
media files). I don't know why.

Fixes: b97cda7 ("mpv-unwrapped: 0.34.1 -> 0.35.0")

Fixes NixOS#205141

Ref jellyfin/jellyfin-media-player#341

(cherry picked from commit 3c528bc)
mweinelt pushed a commit that referenced this pull request Aug 4, 2023
Pull in _FORTIFY_SOURCE=3 stack smashing fix. Without the change on
current `master` `rtorrent` crashes at start as:

*** buffer overflow detected ***: terminated
                                                                                        __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
44      pthread_kill.c: No such file or directory.
(gdb) bt
    #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
    #1  0x00007ffff7880af3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
    #2  0x00007ffff7831c86 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
    #3  0x00007ffff781b8ba in __GI_abort () at abort.c:79
    #4  0x00007ffff781c5f5 in __libc_message (fmt=fmt@entry=0x7ffff7992540 "*** %s ***: terminated\n") at ../sysdeps/posix/libc_fatal.c:150
    #5  0x00007ffff7910679 in __GI___fortify_fail (msg=msg@entry=0x7ffff79924e6 "buffer overflow detected") at fortify_fail.c:24
    #6  0x00007ffff790eea4 in __GI___chk_fail () at chk_fail.c:28
    #7  0x00007ffff790ea85 in ___snprintf_chk (s=<optimized out>, maxlen=<optimized out>, flag=<optimized out>, slen=<optimized out>, format=<optimized out>) at snprintf_chk.c:29
    #8  0x0000000000472acf in utils::Lockfile::try_lock() ()
    #9  0x000000000044b524 in core::DownloadStore::enable(bool) ()
    #10 0x00000000004b1f7b in Control::initialize() ()
    NixOS#11 0x000000000043000b in main ()
mweinelt pushed a commit that referenced this pull request Feb 19, 2024
Since ba83271 the build fails with

    applying patch /nix/store/46rxbbvl2l3mrxb50y9rzy7ahgx0lraj-d741901dddd731895346636c0d3556c6fa51fbe6.patch
    patching file tests/hazmat/primitives/test_aead.py
    Hunk #1 FAILED at 56.
    Hunk #2 FAILED at 197.
    Hunk #3 FAILED at 378.
    Hunk #4 FAILED at 525.
    Hunk #5 FAILED at 700.
    Hunk #6 FAILED at 844.
    6 out of 6 hunks FAILED -- saving rejects to file tests/hazmat/primitives/test_aead.py.rej
mweinelt pushed a commit that referenced this pull request Jul 31, 2024
This adds some extremely helpful and popular encoders in by default:
* openjpeg
* celt
* libwebp
* libaom

On the `master` branch, closure size for ffmpeg-headless went up 18.5 MiB.
```
$ nix store diff-closures nixpkgs#ffmpeg-headless^bin .#ffmpeg-headless^bin
celt: ∅ → 0.11.3, +168.4 KiB
ffmpeg-headless: +70.0 KiB
giflib: ∅ → 5.2.2, +398.7 KiB
lcms2: ∅ → 2.16, +466.2 KiB
lerc: ∅ → 4.0.0, +840.2 KiB
libaom: ∅ → 3.9.0, +8047.8 KiB
libdeflate: ∅ → 1.20, +427.0 KiB
libtiff: ∅ → 4.6.0, +655.9 KiB
libvmaf: ∅ → 3.0.0, +2665.0 KiB
libwebp: ∅ → 1.4.0, +2559.7 KiB
openjpeg: ∅ → 2.5.2, +1525.1 KiB
zstd: ∅ → 1.5.6, +1158.0 KiB

$ nvd diff $(nix build nixpkgs#ffmpeg-headless^bin --print-out-paths --no-link) $(nix build .#ffmpeg-headless^bin --print-out-paths --no-link)
<<< /nix/store/4n60lnj3zkjpasd4c56bzhpx2m8lc1sx-ffmpeg-headless-6.1.1-bin
>>> /nix/store/884f487w5hac6rs94jq6hq5zqkxdv666-ffmpeg-headless-6.1.1-bin
Added packages:
[A.]  #1  celt        0.11.3
[A.]  #2  giflib      5.2.2
[A.]  #3  lcms2       2.16
[A.]  #4  lerc        4.0.0
[A.]  #5  libaom      3.9.0
[A.]  #6  libdeflate  1.20
[A.]  #7  libtiff     4.6.0
[A.]  #8  libvmaf     3.0.0
[A.]  #9  libwebp     1.4.0 x2
[A.]  #10  openjpeg    2.5.2
[A.]  NixOS#11  zstd        1.5.6
Closure size: 66 -> 78 (15 paths added, 3 paths removed, delta +12, disk usage +18.5MiB).
```
mweinelt pushed a commit that referenced this pull request Aug 25, 2024
Strongly inspired by the forgejo counterpart[1], for the following
reasons:

* The feature is broken with the current module and crashes on
  authentication with the following stacktrace (with a PAM service
  `gitea` added):

      server # Stack trace of thread 1008:
      server # #0  0x00007f3116917dfb __nptl_setxid (libc.so.6 + 0x8ddfb)
      server # #1  0x00007f3116980ae6 setuid (libc.so.6 + 0xf6ae6)
      server # #2  0x00007f30cc80f420 _unix_run_helper_binary (pam_unix.so + 0x5420)
      server # #3  0x00007f30cc8108c9 _unix_verify_password (pam_unix.so + 0x68c9)
      server # #4  0x00007f30cc80e1b5 pam_sm_authenticate (pam_unix.so + 0x41b5)
      server # #5  0x00007f3116a84e5b _pam_dispatch (libpam.so.0 + 0x3e5b)
      server # #6  0x00007f3116a846a3 pam_authenticate (libpam.so.0 + 0x36a3)
      server # #7  0x00000000029b1e7a n/a (.gitea-wrapped + 0x25b1e7a)
      server # #8  0x000000000047c7e4 n/a (.gitea-wrapped + 0x7c7e4)
      server # ELF object binary architecture: AMD x86-64
      server #
      server # [   42.420827] gitea[897]: pam_unix(gitea:auth): unix_chkpwd abnormal exit: 159
      server # [   42.423142] gitea[897]: pam_unix(gitea:auth): authentication failure; logname= uid=998 euid=998 tty= ruser= rhost=  user=snenskek

  It only worked after turning off multiple sandbox settings and adding
  `shadow` as supplementary group to `gitea.service`.

  I'm not willing to maintain additional multiple sandbox settings for
  different features, especially given that it was probably not used for
  quite a long time:

  * There was no PR or bugreport about sandboxing issues related to
    PAM.

  * Ever since the module exists, it used the user `gitea`, i.e. it had
    never read-access to `/etc/shadow`.

* Upstream has it disabled by default[2].

If somebody really needs it, it can still be brought back by an overlay
updating `tags` accordingly and modifying the systemd service config.

[1] 07641a9
[2] https://docs.gitea.com/usage/authentication#pam-pluggable-authentication-module
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants