Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random segmentation fault in managed code on 32-bit ARM linux with dotnet 8.0 #102396

Open
maf1024 opened this issue May 17, 2024 · 28 comments
Open
Labels
arch-arm32 area-VM-coreclr tenet-reliability Reliability/stability related issue (stress, load problems, etc.)
Milestone

Comments

@maf1024
Copy link

maf1024 commented May 17, 2024

Description

I'm encountering random segmentation faults (and sometimes AccessViolationException and NullReferenceException) when running a dotnet 8.0 console app on 32-bit ARM linux. It seems to mainly occur for me when it attempts to connect to a SignalR hub as a client.

Reproduction Steps

I have pushed a small repro app pair here.

Note that in addition to the crashing console app, it also contains a trivially simple SignalR webapp that I've deployed to azure when reproducing. (Note the "[CHANGE THIS TO YOUR WEBAPP URL]" line in the console app)

The console app code just tries to make a SignalR connection over websockets and send/receive a few MessagePack messages before exiting.

When running the console app repeatedly, it has a random chance of encountering the issue, after something like 10 to 100 attempts. (a helper bash loop .sh is included)

Expected behavior

Can run repeatedly without crashing.

Actual behavior

Approximately 5% of the time it fails with simply Segmentation fault.

I have uploaded a core crash dump file of one of the segfault occurrences here

On some occasions it randomly has a NullReferenceException, with additional stack trace info:

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.GetStateMachineBox[TStateMachine](TStateMachine& stateMachine, Task`1& taskField)
   at Microsoft.AspNetCore.SignalR.Client.HubConnection.HandshakeAsync(ConnectionState startingConnectionState, Int32 protocolVersion, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsyncCore(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsyncCore(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsyncInner(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsync(CancellationToken cancellationToken)
   at Program.<Main>$(String[] args)
   at Program.<Main>(String[] args)
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AwaitUnsafeOnCompleted[TAwaiter](TAwaiter& awaiter, IAsyncStateMachineBox box)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Fatal error. Internal CLR error. (0x80131506)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_1(System.Object)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()

Regression?

It seems to work fine (even with thousands of attempts) when changing the csproj back to dotnet 6.0 instead of 8.0.

(I can't test dotnet 7.0 on this ARM device due to the higher glibc requirement 7.0 has)

Known Workarounds

Downgrading the csproj to dotnet 6.0

Configuration

Custom linux OS running on 32-bit ARM linux IoT device.

Problem seems specific to 32-bit ARM. Ran repeatedly on a different custom ARM64 device with no issues.

Other information

On some rare occasions with very similar code (from my team's proprietary app), it encountered this AccessViolationException, which I suspect is stemming from the same cause:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ClearStateUponCompletion()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ForceAuthenticationAsync>d__150`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(Int32)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetExistingTaskResult(System.Threading.Tasks.Task`1<Int32>, Int32)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetResult(Int32)
   at System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<ReceiveHandshakeFrameAsync>d__151`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Threading.ThreadPool+<>c.<.cctor>b__52_0(System.Object)
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SignalCompletion()
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetResult(Int32)
   at System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1+StateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1+StateMachineBox`1[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Net.Security.SslStream+<EnsureFullTlsFrameAsync>d__161`1[[System.Net.Security.AsyncReadWriteAdapter, System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]], System.Net.Security, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Threading.ThreadPool+<>c.<.cctor>b__52_0(System.Object)
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SignalCompletion()
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetResult(Boolean)
   at System.Net.Sockets.SocketAsyncEventArgs.OnCompletedInternal()
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAsyncSuccess(Int32, System.Net.Sockets.SocketFlags)
   at System.Net.Sockets.SocketAsyncEventArgs.CompletionCallback(Int32, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError)
   at System.Net.Sockets.SocketAsyncEventArgs.TransferCompletionCallbackCore(Int32, System.Memory`1<Byte>, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError)
   at System.Net.Sockets.SocketAsyncContext+OperationQueue`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ProcessAsyncOperation(System.__Canon)
   at System.Net.Sockets.SocketAsyncContext+ReadOperation.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Net.Sockets.SocketAsyncContext.HandleEvents(SocketEvents)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label May 17, 2024
Copy link
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@antonfirsov antonfirsov removed the untriaged New issue has not been triaged by the area owner label May 18, 2024
@antonfirsov antonfirsov added this to the 9.0.0 milestone May 18, 2024
@jkotas jkotas added the tenet-reliability Reliability/stability related issue (stress, load problems, etc.) label May 18, 2024
@rzikm
Copy link
Member

rzikm commented May 20, 2024

Is it possible to check if your Linux OS flavor uses 64bit time_t? I am aware that couple of distributions moved to 64bit to avoid the year 2038 problem, which creates a binary break for ARM32 platform. This announcement may be relevant dotnet/core#9285.

I have seen our tests suite occasionally crash on ARM32 Debian 12 (on main/9.0), possibly the same issue (but we disabled runs on that specific platform due to #101444 (comment) so I did not investigate further).

@rzikm rzikm self-assigned this May 20, 2024
@maf1024
Copy link
Author

maf1024 commented May 20, 2024

Looking at the result of a printf("size of time_t is: %d\n", sizeof(time_t)); check using our platform-specific gcc and running it on our hardware, it says 4 bytes, so it seems it's 32-bit time_t.

@filipnavara
Copy link
Member

filipnavara commented May 20, 2024

it says 4 bytes, so it seems it's 32-bit time_t.

Unfortunately I am not sure that this test necessarily works. It still depends on the _TIME_BITS build option and the system would somehow have to set it globally and unconditionally (or for the OpenSSL build, specifically).

@maf1024
Copy link
Author

maf1024 commented May 23, 2024

Is there something I can try to run to more conclusively answer the time_t question?

@cw-ametek
Copy link

I think this may be the same issue that I have been tracking down on our ARM32 (nxp imx6q) embedded platform running Linux.

The fault can be reproduced using an unmodified dotnet 8 webapi sample program:

dotnet new webapi -n crash
dotnet publish --self-contained -c Release -f net8.0 -r linux-arm crash

For me the segmentation fault only seems to occur on the first web request, and is much more likely to happen with multiple simultaneous requests.

I've used this script running on the device to automate the process:

#!/bin/bash
PROGRAM=crash
fail=0
RUNS=100

for run in $(seq 1 $RUNS); do
  echo "Run $run ($fail failures)"
  ./$PROGRAM > /dev/null &
  sleep 5
  curl -k --parallel --parallel-immediate --parallel-max 50  http://localhost:5000/weatherforecast?[1-16] > /dev/null
  sleep 5

  PID=$(pidof $PROGRAM)
  if [ "$PID" == "" ]; then
    echo "CRASHED!"
        ((fail++))
  else
        kill $PID
        sleep 2
  fi
done

echo "Program crashes $fail / $RUNS"

With a single request, the failure rate is only about 3%, but goes up to about 25% with 4 or more requests.

.net7 shows the same fault, but .net6 works without any problems.
ARM64 and Ubuntu in WSL did not show the problem.

The kernel (5.4.147) is compiled with CONFIG_64BIT_TIME, but glibc is only version 2.31
I can't see any indication that _TIME_BITS is being defined anywhere, so I think it is just 32 bit.

I can collect a coredump if that will help.

Sometimes I get the AccessViolationException instead of a segmentation fault:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__8 ByRef)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1[[Microsoft.AspNetCore.Mvc.ModelBinding.ModelBindingResult, Microsoft.AspNetCore.Mvc.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__8 ByRef)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(Microsoft.AspNetCore.Mvc.ActionContext, Microsoft.AspNetCore.Mvc.ModelBinding.IModelBinder, Microsoft.AspNetCore.Mvc.ModelBinding.IValueProvider, Microsoft.AspNetCore.Mvc.Abstractions.ParameterDescriptor, Microsoft.AspNetCore.Mvc.ModelBinding.ModelMetadata, System.Object, System.Object)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0.<CreateBinderDelegate>g__Bind|0(Microsoft.AspNetCore.Mvc.ControllerContext, System.Object, System.Collections.Generic.Dictionary`2<System.String,System.Object>)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.BindArgumentsAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Mvc.Routing.ControllerRequestDelegateFactory+<>c__DisplayClass12_0.<CreateRequestDelegate>b__0(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.SetRoutingAndContinue(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecuteFromThreadPool(System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Aborted


Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetException(System.Exception, System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult> ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetException(System.Exception)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection`1+<ExecuteAsync>d__8[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection`1+<ExecuteAsync>d__8[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection`1+<ExecuteAsync>d__8[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection`1+<ExecuteAsync>d__8[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetExistingTaskResult(System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>, System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult()
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.HttpConnection+<ProcessRequestsAsync>d__12`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.HttpConnection+<ProcessRequestsAsync>d__12`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.HttpConnection+<ProcessRequestsAsync>d__12`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.HttpConnection+<ProcessRequestsAsync>d__12`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetExistingTaskResult(System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>, System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult()
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequestsAsync>d__237`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequestsAsync>d__237`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequestsAsync>d__237`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequestsAsync>d__237`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Runtime.CompilerServices.IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(System.Object)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TrySetResult(System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].SetExistingTaskResult(System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>, System.Threading.Tasks.VoidTaskResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult()
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__238`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__238`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__238`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__238`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.AspNetCore.Server.Kestrel.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecuteFromThreadPool(System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Aborted

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Linq.Expressions.Compiler.VariableStorageKind, System.Linq.Expressions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]]..ctor(Int32, System.Collections.Generic.IEqualityComparer`1<System.__Canon>)
   at System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Linq.Expressions.Compiler.VariableStorageKind, System.Linq.Expressions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]]..ctor()
   at System.Text.Json.Serialization.Metadata.ReflectionEmitMemberAccessor.CreatePropertyGetter[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Reflection.PropertyInfo)
   at System.Text.Json.Serialization.Metadata.ReflectionEmitCachingMemberAccessor+<>c__12`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<CreatePropertyGetter>b__12_0(System.ValueTuple`3<System.String,System.Type,System.Reflection.MemberInfo>)
   at System.Text.Json.Serialization.Metadata.ReflectionEmitCachingMemberAccessor+Cache`1+<>c__6`1[[System.ValueTuple`3[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<GetOrAdd>b__6_0(System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>, System.Func`2<System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>,System.__Canon>)
   at System.Collections.Concurrent.ConcurrentDictionary`2[[System.ValueTuple`3[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetOrAdd[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>, System.Func`3<System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>,System.__Canon,System.__Canon>, System.__Canon)
   at System.Text.Json.Serialization.Metadata.ReflectionEmitCachingMemberAccessor+Cache`1[[System.ValueTuple`3[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetOrAdd[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>, System.Func`2<System.ValueTuple`3<System.__Canon,System.__Canon,System.__Canon>,System.__Canon>)
   at System.Text.Json.Serialization.Metadata.ReflectionEmitCachingMemberAccessor.CreatePropertyGetter[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Reflection.PropertyInfo)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.DeterminePropertyAccessors[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1<System.__Canon>, System.Reflection.MemberInfo, Boolean)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].DetermineReflectionPropertyAccessors(System.Reflection.MemberInfo, Boolean)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.PopulatePropertyInfo(System.Text.Json.Serialization.Metadata.JsonPropertyInfo, System.Reflection.MemberInfo, System.Text.Json.Serialization.JsonConverter, System.Nullable`1<System.Text.Json.Serialization.JsonIgnoreCondition>, Boolean, Boolean)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.CreatePropertyInfo(System.Text.Json.Serialization.Metadata.JsonTypeInfo, System.Type, System.Reflection.MemberInfo, System.Text.Json.JsonSerializerOptions, Boolean, Boolean)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.AddMembersDeclaredBySuperType(System.Text.Json.Serialization.Metadata.JsonTypeInfo, System.Type, Boolean, PropertyHierarchyResolutionState ByRef)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.PopulateProperties(System.Text.Json.Serialization.Metadata.JsonTypeInfo)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.CreateTypeInfoCore(System.Type, System.Text.Json.Serialization.JsonConverter, System.Text.Json.JsonSerializerOptions)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.GetTypeInfo(System.Type, System.Text.Json.JsonSerializerOptions)
   at System.Text.Json.JsonSerializerOptions.GetTypeInfoNoCaching(System.Type)
   at System.Text.Json.JsonSerializerOptions+CachingContext.CreateCacheEntry(System.Type, CachingContext)
   at System.Collections.Concurrent.ConcurrentDictionary`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetOrAdd[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.Func`3<System.__Canon,System.__Canon,System.__Canon>, System.__Canon)
   at System.Text.Json.JsonSerializerOptions.GetTypeInfoInternal(System.Type, Boolean, System.Nullable`1<Boolean>, Boolean, Boolean)
   at System.Text.Json.JsonSerializerOptions.GetTypeInfo(System.Type)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter+<WriteResponseBodyAsync>d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter+<WriteResponseBodyAsync>d__5, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<WriteResponseBodyAsync>d__5 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter+<WriteResponseBodyAsync>d__5, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<WriteResponseBodyAsync>d__5 ByRef)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext, System.Text.Encoding)
   at Microsoft.AspNetCore.Mvc.Formatters.TextOutputFormatter.WriteAsync(Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor.ExecuteAsyncCore(Microsoft.AspNetCore.Mvc.ActionContext, Microsoft.AspNetCore.Mvc.ObjectResult, System.Type, System.Object)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor.ExecuteAsync(Microsoft.AspNetCore.Mvc.ActionContext, Microsoft.AspNetCore.Mvc.ObjectResult)
   at Microsoft.AspNetCore.Mvc.ObjectResult.ExecuteResultAsync(Microsoft.AspNetCore.Mvc.ActionContext)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultAsync(Microsoft.AspNetCore.Mvc.IActionResult)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAsync[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeAsync>g__Logged|17_1>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeAsync>g__Logged|17_1>d, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<InvokeAsync>g__Logged|17_1>d ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeAsync>g__Logged|17_1>d, Microsoft.AspNetCore.Mvc.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<InvokeAsync>g__Logged|17_1>d ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Mvc.Routing.ControllerRequestDelegateFactory+<>c__DisplayClass12_0.<CreateRequestDelegate>b__0(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.SetRoutingAndContinue(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext(System.Threading.Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware+<<Invoke>g__AwaitMatcher|10_0>d, Microsoft.AspNetCore.Routing, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].ExecuteFromThreadPool(System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Aborted 

@jkotas
Copy link
Member

jkotas commented May 24, 2024

This looks like a GC hole.

Copy link
Contributor

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

@jkotas jkotas added the untriaged New issue has not been triaged by the area owner label May 24, 2024
@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label May 24, 2024
@mangod9
Copy link
Member

mangod9 commented Jul 10, 2024

Hey @maf1024 @cw-ametek would you be able to provide a dump of the failure? If this consistently repros on certain platform might be worth trying with DOTNET__HeapVerify=1 which might help diagnose heap corruption.

@maf1024
Copy link
Author

maf1024 commented Jul 10, 2024

@mangod9 Buried in my original wall of text was a link to one sample of a crash dump (hosted on my free MSDN credits 😊)

I'll try to attempt with HeapVerify soon and see if it makes the failures more consistent. Would it also possibly yield a more useful crash dump? (I'm unfamiliar with that option but it sounds like it does more exhaustive policing of memory)

@maf1024
Copy link
Author

maf1024 commented Jul 11, 2024

@mangod9 DOTNET__HeapVerify=1 doesn't seem to change the randomness of failures on my platform. Here's a batch of dumps with that setting enabled.

@mangod9
Copy link
Member

mangod9 commented Jul 11, 2024

Ok, we will investigate. However, just realized that I had a typo in the env. var DOTNET_HeapVerify=1 (single underscore not double :( )

@mangod9
Copy link
Member

mangod9 commented Jul 12, 2024

The dumps are not easily diagnosable since this is on a custom distro. Assume you are not able to repro on standard Debian/Ubuntu/Alpine ?

@maf1024
Copy link
Author

maf1024 commented Jul 15, 2024

I don't have any other 32-bit ARM hardware to even test it out on.

I'm open to suggestions for virtualized approaches to repro in a more standardized way.

@maf1024
Copy link
Author

maf1024 commented Jul 15, 2024

Also for what it's worth, DOTNET_HeapVerify=1 when set correctly doesn't seem to change the randomness of the failures (and I did see a noticeable performance drop, indicating I set it correctly)

@Olaf-Kober-AP
Copy link

We noticed the same issue while porting our .NET applications from Mono to .NET 7/8. Since we could not find a quick solution, we stayed on .NET 6 first, as .NET 6 does not show the same crashes.

Note, our applications run on NXP iMX6 dual/quad cores (Cortex-A9 ARM32) with a custom Yocto-based Linux (kernel 5.15).

We initially observed random seg faults while running "dotnet test" (we run unit tests on our embedded target hardware), but were able to reproduce those seg faults also by running PowerShell Core, for example, a simple command like

while true; do dotnet .pwsh/pwsh.dll -c ls; done

seg faults within a few minutes. Even simple applications like dotnet --info crash, although it happens less often.

But the repo sample posted by @cw-ametek (#102396 (comment)) might be easier to debug. We verified it on our hw/os combination; it crashes almost immediately.

We tried different hardware and custom operating system versions in the past. For example, a Raspberry PI 3 with a Ubuntu 32.04 Server Edition, 32bit does not show these crashes. Also other hardware based on Cortex-A15 does not seg fault. Is this related to Cortex-A9 only? We are not sure yet.

Disabling Ready2Run, Tier Compilation, and using different GC settings (server, without concurrent, background GC, etc.) does not change anything. But, reducing the number of CPU cores helps. For example, on a dual-core system the crashes don't happen that often, also, when binding the process via taskset to a single CPU core reduces the probability of these seg faults to almost zero.

That said, would it help if we provide crash dumps? We are also on a custom Linux distro, but we can provide the necessary symbols for debugging those dumps, if needed.

Can we do anything else? As a company we have high interest in getting that fixed.

@michaldobrodenka
Copy link

I'm using dotnet on ARM32 IoT devices in thousands, dotnet process uptime in months, without problems. My CPU is Allwinner H3, 4x ARM A7 on custom Yocto Linux. I have also monitoring of the nodes, so I see if there is process restart or any problem. I migrated from mono to dotnet 5.0 and now I'm using latest 8.0 and kernel 5.15.35. My app is not small, about 1M of lines of code + external libraries.

@jkotas
Copy link
Member

jkotas commented Jul 25, 2024

That said, would it help if we provide crash dumps?

Could you please share stacktrace of the crash to start with?

@Olaf-Kober-AP
Copy link

Sure!

Here are a few managed-only stack traces that we recorded in the past (various applications).

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Collections.Generic.Dictionary`2[[Interop+Error, System.Net.Sockets, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[System.Net.Sockets.SocketError, System.Net.Primitives, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].TryInsert(Error, System.Net.Sockets.SocketError, System.Collections.Generic.InsertionBehavior)
   at System.Net.Sockets.SocketErrorPal.GetSocketErrorForNativeError(Error)
   at System.Net.Sockets.SafeSocketHandle.DoCloseHandle(Boolean)
   at System.Net.Sockets.SafeSocketHandle.CloseHandle(Boolean, Boolean)
   at System.Net.Sockets.SafeSocketHandle.CloseAsIs(Boolean)
   at System.Net.Sockets.Socket.Dispose(Boolean)
   at System.Net.Sockets.Socket.Dispose()
   at NetMQ.Core.Utils.Signaler..ctor()
   at NetMQ.Core.Mailbox..ctor(System.String)
   at NetMQ.Core.Ctx..ctor()
   at NetMQ.NetMQConfig.get_Context()
   at NetMQ.NetMQSocket..ctor(NetMQ.ZmqSocketType, System.String, DefaultAction)
   at NetMQ.Sockets.PairSocket..ctor(System.String)
   at NetMQ.Sockets.PairSocket.CreateSocketPair(NetMQ.Sockets.PairSocket ByRef, NetMQ.Sockets.PairSocket ByRef, System.Action`1<NetMQ.Sockets.PairSocket>, System.Action`1<NetMQ.Sockets.PairSocket>)
   at NetMQ.NetMQQueue`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]..ctor(Int32)
   at Darwin.Shared.Monitoring.Transport.MonitoringPublisher..ctor(Int32, Int32, System.TimeSpan, System.TimeSpan, PostSharp.Aspects.ConstructorDepth)
   at Darwin.Shared.Monitoring.Transport.MonitoringPublisher..ctor(Int32, PostSharp.Aspects.ConstructorDepth)
   at Darwin.Shared.Monitoring.Transport.MonitoringPublisher..ctor(Int32)
   at Darwin.Shared.Monitoring.MonitoringServer+<StartAsync>d__39.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Darwin.Shared.Monitoring.MonitoringServer.StartAsync(Boolean, Boolean)
   at Darwin.Fabric.Systeme.Initializers.SystemMonitoringServerInitializer+<_StartServer>d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Darwin.Fabric.Systeme.Initializers.SystemMonitoringServerInitializer._StartServer(Darwin.Fabric.Runtime.IRuntimeData, Darwin.Shared.Monitoring.IMonitoringServer)
   at Darwin.Fabric.Systeme.Initializers.SystemMonitoringServerInitializer+<>c__DisplayClass2_0+<<_SetupCounters>b__0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Darwin.Fabric.Systeme.Initializers.SystemMonitoringServerInitializer+<>c__DisplayClass2_0.<_SetupCounters>b__0()
   at System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].InnerInvoke()
   at System.Threading.Tasks.Task+<>c.<.cctor>b__281_0(System.Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
   at System.Threading.Tasks.Task.ExecuteFromThreadPool(System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Runtime.CompilerServices.CastHelpers.IsInstanceOfInterface(Void*, System.Object)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections.Generic.IEnumerable`1<Darwin.Fabric.PluginId>)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections.Generic.IEnumerable`1<Darwin.Fabric.PluginId>)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections.Generic.IEnumerable`1<Darwin.Fabric.PluginId>)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections.Generic.IEnumerable`1<Darwin.Fabric.PluginId>)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections.Generic.IEnumerable`1<Darwin.Fabric.PluginId>)
   at System.Linq.Enumerable.ToArray[[Darwin.Fabric.PluginId, Darwin.Fabric, Version=6.1.0.0, Culture=neutral, PublicKeyToken=ab309ed73cde3b9e]](System.Collections
...
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at System.Collections.Generic.HashSet`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].AddIfNotPresent(System.__Canon, Int32 ByRef)
   at System.Collections.Generic.HashSet`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Add(System.__Canon)
   at Microsoft.DotNet.Cli.Telemetry.TelemetryFilter.get_ParseResultLogRules()
   at Microsoft.DotNet.Cli.Telemetry.TelemetryFilter.get_ParseResultLogRules()
   at Microsoft.DotNet.Cli.Program.ProcessArgs(System.String[], System.TimeSpan, Microsoft.DotNet.Cli.Telemetry.ITelemetry)
   at Microsoft.DotNet.Cli.Program.Main(System.String[])
Process terminated. An exception was not handled in an AsyncLocal<T> notification callback.
   at System.Environment.FailFast(System.String, System.Exception)
   at System.Threading.ExecutionContext.OnValuesChanged(System.Threading.ExecutionContext, System.Threading.ExecutionContext)
   at System.Threading.ExecutionContext.RestoreChangedContextToThread(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ExecutionContext)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
   at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object)
   at System.Threading.Thread+StartHelper.RunWorker()
   at System.Threading.Thread+StartHelper.Run()
   at System.Threading.Thread.StartCallback()
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Threading.ExecutionContext.OnValuesChanged(ExecutionContext previousExecutionCtx, ExecutionContext nextExecutionCtx)

We see very erratic exception patterns. Looks like a kind of memory corruption to me.

Right now, a colleague is trying to get managed+native stack traces via lldb+sos from one of the crash dumps, but we are facing some problems. The thread causing the crash (seg fault) does not show any backtraces. Maybe the dump is corrupted, too. We are working on it and come back to you.

Sidenote: I have to hurry into vacation. My colleagues will take over.

@lvorpahl-nokia
Copy link

We see the same random segmentation faults with our cross built armel dotnet 8.0 runtime.
Our applications run on a Marvell Armada XP SoC (ARMv7 CPU) with a custom Yocto-based Linux (kernel 5.10).
Previously we used dotnet 6.0 where we did not see the problem.

@jkotas
Copy link
Member

jkotas commented Sep 18, 2024

@lvorpahl-nokia Are you able to collect a crash dump and open it under lldb w/ SOS extension? The next step to diagnose these crashes is to run VerifyHeap SOS command on the crash dump to see whether the GC heap is corrupted.

@michaldobrodenka
Copy link

Marvell Armada XP SoC is based on Sheeva PJ4B-MP cores, which are probably modified Cortex A8. The other cpu cores with this problem were Cortex A9. I don't know if it is relevant. A8 & A9 are older designs based on ARMv7 ISA.

I have no problems with ARM A7 cores. Could there be a connection with different FPU (A8 & A9 have VFPv3)? More recent deisgns use VFPv4 (A7/15/17...)

@AndreyLalaev
Copy link
Contributor

@michaldobrodenka, hmm, it is an interesting point. Thank you!

Looks like by default coreclr for ARM32 is compiled with 32 64-bits FPU support(link to doc), but Cortex-A9 uses VFPv3-D16.
I will recompile coreclr for different FPU and retest tomorrow.

@lvorpahl-nokia
Copy link

lvorpahl-nokia commented Sep 19, 2024

@jkotas Unfortunately no, our toolchain has no lldb support. Therefore I apologize that I can not contribute more than reporting that we see the crashes on our CPU. We compiled coreclr with VFPv3-D16.

@michaldobrodenka
Copy link

@jkotas Unfortunately no, our toolchain has no lldb support. Therefore I apologize that I can not contribute more than reporting that we see the crashes on our CPU. We compiled coreclr with VFPv3-D16.

Sorry for off topic, but what is thre reason you are compiling coreclr by yourself? Still somewhere could be VPFv4 assembly? But small chance probably :(

@lvorpahl-nokia
Copy link

We are running an armel Linux and there is no official armel build as far as I know.

@michaldobrodenka
Copy link

michaldobrodenka commented Sep 19, 2024

I'm getting lost in these arm arch and abbreviations, but I'm compiling dotnet for ARMv6, use mono-vm and it's usable on R Pi1. It's not core-clr, but it might be helpful.

I'm using it as a docker container to build self-contained executable which then runs on Raspberry Pi compute modules. Only downside is, that every timer will stop working after UInt32.Max millis (49.7days).

If you want to try it: https://hub.docker.com/r/taphome/dotnet-armv6

@michaldobrodenka
Copy link

michaldobrodenka commented Sep 19, 2024

And you can still use dotnet with mono runtime for ARMv7, it might help, use -p:UseMonoRuntime=true -p:SelfContained=true when compiling your app

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-arm32 area-VM-coreclr tenet-reliability Reliability/stability related issue (stress, load problems, etc.)
Projects
None yet
Development

No branches or pull requests