OSX: openmp crash with macports libomp12.0.0

[MStraeten]

not really a darktable bug, but worth to have in mind when building darktable on osx, maybe worth to ckeck if thats also on other platforms using libomp12:

opening an image in darkroom crashes, if darktable is linked with libomp12.0.0 (libomp11.1.0.0 is fine)

OMP: Error #131: Thread identifier invalid.
Process 27644 stopped
* thread #72, stop reason = signal SIGABRT
    frame #0: 0x00007fff204a0946 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff204a0946 <+10>: jae    0x7fff204a0950            ; <+20>
    0x7fff204a0948 <+12>: movq   %rax, %rdi
    0x7fff204a094b <+15>: jmp    0x7fff2049ab49            ; cerror_nocancel
    0x7fff204a0950 <+20>: retq
Target 0: (darktable) stopped.
(lldb) bt
* thread #72, stop reason = signal SIGABRT
  * frame #0: 0x00007fff204a0946 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff204cf615 libsystem_pthread.dylib`pthread_kill + 263
    frame #2: 0x00007fff20424411 libsystem_c.dylib`abort + 120
    frame #3: 0x00000001005b0d29 libomp.dylib`__kmp_abort_process + 48
    frame #4: 0x00000001005af821 libomp.dylib`__kmp_fatal + 147
    frame #5: 0x00000001005d929e libomp.dylib`__kmpc_for_static_init_4 + 1412
    frame #6: 0x0000000100269668 libdarktable.dylib`.omp_outlined..2 [inlined] .omp_outlined._debug__.1(.global_tid.=<unavailable>, .bound_tid.=<unavailable>, roi_out=0x000070000164e070, out=<unavailable>, out_stride=<unavailable>, px_footprint=<unavailable>, roi_in=<unavailable>, rggby=<unavailable>, rggbx=<unavailable>, filters=<unavailable>, in=<unavailable>, clut=<unavailable>, in_stride=<unavailable>) at imageop_math.c:208:1 [opt]
    frame #7: 0x000000010026960b libdarktable.dylib`.omp_outlined..2(.global_tid.=<unavailable>, .bound_tid.=<unavailable>, roi_out=0x000070000164e070, out=0x0000000137dd6040, out_stride=1362, px_footprint=0, roi_in=0x000070000164e048, rggby=0, rggbx=0, filters=0, in=0x000000013cfe6040, clut=0x000070000164df90, in_stride=0) at imageop_math.c:208 [opt]
    frame #8: 0x00000001005eae93 libomp.dylib`__kmp_invoke_microtask + 147
    frame #9: 0x00000001005b6a6f libomp.dylib`__kmp_invoke_task_func + 165
    frame #10: 0x00000001005b5ec7 libomp.dylib`__kmp_launch_thread + 226
    frame #11: 0x00000001005dc992 libomp.dylib`__kmp_launch_worker(void*) + 278
    frame #12: 0x00007fff204cf954 libsystem_pthread.dylib`_pthread_start + 224
    frame #13: 0x00007fff204cb4a7 libsystem_pthread.dylib`thread_start + 15
(lldb)

imageop_math.c:208:

#ifdef _OPENMP
#pragma omp parallel for default(none) \
  dt_omp_firstprivate(filters, in, in_stride, out, out_stride, px_footprint, rggbx, rggby, roi_in, roi_out) \
  shared(clut) schedule(static)
#endif

a further crash (using a jpg) interpolation.c:1622:

#ifdef _OPENMP
#pragma omp parallel for default(none) \
  dt_omp_firstprivate(in, in_stride, out_stride, roi_out) \
  shared(out, hindex, hlength, hkernel, vindex, vlength, vkernel, vmeta)
#endif

To Reproduce
build with latest macports with libomp12.0.0

Which commit introduced the error
definitely not introduced with a specific commit - i built an older revision of current master and this also crashed.
reactivating libomp11.1.0.0 results in a build that runs properly

Platform

• darktable version : darktable 3.5.0+2228~gabfd8d42e
• OS : OSX built with XCode 12.5, sdk 11.3

[github-actions]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[MStraeten]

seems to be no darktable issue but an libomp12 issue.
so just for information.

[parafin]

I've built darktable with libomp 12 and don't get a crash. Is there anything specific I should do to reproduce the issue?

[MStraeten]

unfortunately it's just opening an image in darkroom that results in an crash if omp parallelization is affected
my steps to get a reliable environment:

• sudo port activate libomp --> select 2) libomp @12.0.0_0
• purge darktable from /usr/*
• purge darktable build directory
• initial cmake configuration

cmake .. \
 -DCMAKE_OSX_DEPLOYMENT_TARGET=10.15 \
 -DCMAKE_CXX_FLAGS=-stdlib=libc++ \
 -DCMAKE_OBJCXX_FLAGS=-stdlib=libc++ \
 -DOpenMP_C_INCLUDE_DIR=/opt/local/include/libomp \
 -DOpenMP_CXX_INCLUDE_DIR=/opt/local/include/libomp \
 -DCMAKE_LIBRARY_PATH=/opt/local/lib/libomp \
 -DBINARY_PACKAGE_BUILD=ON \
 -DRAWSPEED_ENABLE_LTO=ON \
 -DBUILD_CURVE_TOOLS=ON \
 -DBUILD_NOISE_TOOLS=ON \
 -DCMAKE_BUILD_TYPE=Release

build darktable and then run ./bin/darktable --disable-opencl from build directory
same result with -DCMAKE_BUILD_TYPE=RelWithDebInfo

even after uninstalling all libomp versions from macports and reinstalling it from source ...

quite strange:
a darktable build with just libomp @12.0 being available in macports /opt/local/... and then build and activate libomp@11 without rebuilding darktable is fine

[parafin]

I’m guessing then it’s either a bug in new libomp exposed by new XCode or a bug in new XCode exposed by new libomp, because I’m not seeing it with older XCode…

[MStraeten]

so i'll check when the official package is out ;)

[parafin]

I've rechecked my build on the latest macOS (and M1 Mac) and indeed it crashes. So I guess it's not about Xcode version, but rather macOS version. I'll try reverting to older libomp as you suggesting.

[MStraeten]

since this is is no darktable issue and the osx build instructions are updated to deal with this, this issue can be closed.

[parafin]

I confirm that libomp 11.1.0 works fine on M1 Mac. Should we maybe report this issue upstream somewhere (macports or libomp)?..

[johnny-bit]

I think both places need to know about the issue.

[MStraeten]

with libomp 12.0.1 the issue is no longer reproducible with latest Xcode and sdk

[parafin]

but it's reproducible for me built on Xcode 11.3.1 and run on macOS 11

[MStraeten]

then it's better to keep the downgrade ...

[MStraeten]

libomp13.0.0 is out - crash not reproducible with build using Xcode13, sdk12 (macports deployment target 10.15) and darktable build with additional option
-DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
and also Xcode12 + sdk10.15

[parafin]

Seems that indeed crash was fixed, no need for libomp downgrade, amended BUILD.txt accordingly.