Bus error 10 on 1.8.2 aarch64

[evetion]

Since 1.8.2 on Mac M1 (aarch64) the following snippet will crash Julia:

using GeoArrays
ga = GeoArray(rand(24, 24))
GeoArrays.write("test_cog.tif", ga, shortname="COG")

with

signal (10): Bus error: 10
in expression starting at test_crash.jl:3
unknown function (ip: 0x14aae0b08)
Allocations: 16899091 (Pool: 16892413; Big: 6678); GC: 12

In 1.8.1 (and 1.8.2 on other (CI) platforms) this works. I will try to debug this further, under the hood (via ArchGDAL, GDAL, GDAL_jll) this ccall an external C++ library.

Might relate to #47171.

[evetion]

LLDB output: https://gist.github.com/evetion/9f6a35d33d04ea26b16c370a7e89b267

[gbaraldi]

It does seem to come from inside libnetcdf, so it might not be julia.

[evetion]

I agree the backtrace looks suspect, but this code, and the same library, works correctly on previous Julia versions.

[gbaraldi]

The same code works on 1.8.1 on aarch-64 mac but not on 1.8.2.
Was able to reproduce the failure. Will Bisect.

[gbaraldi]

Bisected to #45173. @JeffBezanson

[gbaraldi]

On master I don't get a bus error but I do get segfault.

[gbaraldi]

I went a lot deeper into this and the issue seems to be that we used to this ccall with the following arguments

Hit breakpoint:
In gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7) at /Users/gabrielbaraldi/.julia/packages/GDAL/TCvFp/src/libgdal.jl:6232
 6232  function gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7)
>6233      aftercare(
 6234          ccall(
 6235              (:GDALCreateCopy, libgdal),
 6236              GDALDatasetH,
 6237              (

About to run: (Base.cconvert)(Ptr{Nothing}, Ptr{Nothing} @0x0000600001eeca90)
1|debug> fr
[1] gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7) at /Users/gabrielbaraldi/.julia/packages/GDAL/TCvFp/src/libgdal.jl:6232
  | arg1::Ptr{Nothing} = Ptr{Nothing} @0x0000600001eeca90
  | arg2::String = "test_cog.tif"
  | arg3::Ptr{Nothing} = Ptr{Nothing} @0x0000000135e32140
  | arg4::Bool = false
  | arg5::Vector{String} = String[]
  | arg6::Ptr{Nothing} = Ptr{Nothing} @0x0000000000000000
  | arg7::Ptr{Nothing} = Ptr{Nothing} @0x0000000000000000

Note that arg6 is a NULL.
On 1.8.2 we now get

1|debug> n
In gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7) at /Users/gabrielbaraldi/.julia/packages/GDAL/TCvFp/src/libgdal.jl:6232
 6232  function gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7)
>6233      aftercare(
 6234          ccall(
 6235              (:GDALCreateCopy, libgdal),
 6236              GDALDatasetH,
 6237              (

About to run: <(JuliaInterpreter.CompiledCalls.var"##compiled_ccall#347")(Ptr{Nothing} @0x0000600001b2c270, Cstring(...>
1|debug> fr
[1] gdalcreatecopy(arg1, arg2, arg3, arg4, arg5, arg6, arg7) at /Users/gabrielbaraldi/.julia/packages/GDAL/TCvFp/src/libgdal.jl:6232
  | arg1::Ptr{Nothing} = Ptr{Nothing} @0x0000600001b2c270
  | arg2::String = "test_cog.tif"
  | arg3::Ptr{Nothing} = Ptr{Nothing} @0x000000013d625180
  | arg4::Bool = false
  | arg5::Vector{String} = String[]
  | arg6::Ptr{Nothing} = Ptr{Nothing} @0x0000000151cf4b08
  | arg7::Ptr{Nothing} = Ptr{Nothing} @0x0000000000000000

Now note that arg6 is not null.

From the Macos crash reporter I get

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x0000000151cf4b08
Exception Codes:       0x0000000000000002, 0x0000000151cf4b08

Termination Reason:    Namespace SIGNAL, Code 10 Bus error: 10
Terminating Process:   exc handler [17200]

Which is the exact pointer that we passed to arg6.

[evetion]

And the sixth argument is the user provided callback progress function I believe, which doesn't work on m1. So we did the following:

https://github.com/yeesian/ArchGDAL.jl/blob/49e5a32986b8d209e63d9cfc6b566d49f9f01276/src/utils.jl#L231

[gbaraldi]

I'm confused as to why on 1.8.1 I get a NULL, and the other I get a an actual pointer.

Is passing a NULL to gdal expected @evetion ?

[visr]

Passing a NULL is valid if you don't want a progress callback. That is probably what the workaround in ArchGDAL should enforce rather that the dummy cfunction it gets now.

[gbaraldi]

Oh, so it was a bug that it was getting a NULL pointer then, and that change made it so it actually got the correct results in 1.8.2, which broke it :)

[evetion]

Hmm, odd, because on 1.8.1 I do get a pointer back:

julia> using ArchGDAL
julia> @cfunction(
                   ArchGDAL.GDAL.gdaldummyprogress,
                   Cint,
                   (Cdouble, Cstring, Ptr{Cvoid})
               )
Ptr{Nothing} @0x000000013fae007c
julia> Sys.ARCH == :aarch64
true

[gbaraldi]

You have to get it from deep in there, by modifying the code. Which is why I think there was a bug.
Add an print in gdalcreatecopy or archgdal.copy

[evetion]

Thanks for digging in! Nasty bug, I've made a PR over at ArchGDAL to fix it.

My summary:

• ArchGDAL has a bugged macro for :aarch64 that should return a cfunction pointer, but didn't (just Ptr{Nothing} @0x0000000000000000).
• Julia >=1.8.2 changed the behavior of the bugged macro and this made it return an actual (non-zero) pointer to somewhere, which is this crash report issue.