1.19.2 Support for CUDA 12.6 and CUDNN 9.4

[K4HVH]

Describe the issue

Hello all,

I'm currently having issues getting CUDA 12.6 (Update 1) and CUDNN 9.4.0 to load. I'm getting crashes on a sessionOptions.AppendExecutionProvider_CUDA(cudaOption); call. Are these latest versions of CUDA and CUDNN supported, I have previously used 1.18 with CUDA 11 and it worked a charm, only since attempting to upgrade to 1.19.2 am I getting these issues.

'focus.exe' (Win32): Loaded 'C:\Users\brone\Documents\Programming\Focus\focus-cpp\x64\Debug\onnxruntime_providers_shared.dll'.
'focus.exe' (Win32): Loaded 'C:\Users\brone\Documents\Programming\Focus\focus-cpp\x64\Debug\onnxruntime_providers_cuda.dll'.
'focus.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cublas64_12.dll'. Module was built without symbols.
'focus.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cufft64_11.dll'. Module was built without symbols.
'focus.exe' (Win32): Loaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cublasLt64_12.dll'. Module was built without symbols.
'focus.exe' (Win32): Unloaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cublasLt64_12.dll'
'focus.exe' (Win32): Unloaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cublas64_12.dll'
'focus.exe' (Win32): Unloaded 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\cufft64_11.dll'
'focus.exe' (Win32): Unloaded 'C:\Users\brone\Documents\Programming\Focus\focus-cpp\x64\Debug\onnxruntime_providers_cuda.dll'
Exception thrown at 0x00007FFCC7B0FABC in focus.exe: Microsoft C++ exception: onnxruntime::OnnxRuntimeException at memory location 0x000000C99B31DD20.
Exception thrown at 0x00007FFCC7B0FABC in focus.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
Exception thrown at 0x00007FFCC7B0FABC in focus.exe: Microsoft C++ exception: onnxruntime::OnnxRuntimeException at memory location 0x000000C99B31DD20.
Exception thrown at 0x00007FFCC7B0FABC in focus.exe: Microsoft C++ exception: Ort::Exception at memory location 0x000000C99B31DEA0.
Debug Error!

Above is part of the debug log, it seems to be loading 12.6 fine. I can provide a full log if needed

To reproduce

You can use my ONNXRuntime YOLO inferencer library to test (https://github.com/K4HVH/YOLOv8-ONNXRuntime-CPP)

Install the latest CUDA and CUDNN from https://developer.nvidia.com/

Compile against ONNXRuntime 1.19.2 libs

Program crashes when attempting to switch to CUDA execution provider

Urgency

No response

Platform

Windows

OS Version

Windows 11 Pro - 10.0.22631 Build 22631

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

ONNX Runtime API

C++

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 12.6 - CUDNN 9.4