ONNX integration doesn't want to work on M1

[gbaranski]

Fails on

val model = ONNXModels.ObjectDetection.SSD.pretrainedModel(modelHub)

Exception in thread "main" java.lang.UnsatisfiedLinkError: no onnxruntime in java.library.path: /Users/gbaranski/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:848)
        at java.base/java.lang.System.loadLibrary(System.java:2015)
        at ai.onnxruntime.OnnxRuntime.load(OnnxRuntime.java:201)
        at ai.onnxruntime.OnnxRuntime.init(OnnxRuntime.java:106)
        at ai.onnxruntime.OrtEnvironment.<clinit>(OrtEnvironment.java:25)
        at org.jetbrains.kotlinx.dl.api.inference.onnx.OnnxInferenceModel$Companion.initializeONNXModel$onnx(OnnxInferenceModel.kt:67)
        at org.jetbrains.kotlinx.dl.api.inference.loaders.ONNXModelHub.loadModel(ONNXModelHub.kt:60)
        at org.jetbrains.kotlinx.dl.api.inference.keras.loaders.ModelHub.loadModel$default(ModelHub.kt:45)
        at org.jetbrains.kotlinx.dl.api.inference.onnx.ONNXModels$ObjectDetection$SSD.pretrainedModel(ONNXModels.kt:194)
        at com.gbaranski.plugins.RoutingKt.configureRouting(Routing.kt:18)
        at com.gbaranski.ApplicationKt$main$1.invoke(Application.kt:9)
        at com.gbaranski.ApplicationKt$main$1.invoke(Application.kt:8)
        at io.ktor.server.engine.internal.CallableUtilsKt.executeModuleFunction(CallableUtils.kt:51)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:329)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:328)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:353)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.launchModuleByName(ApplicationEngineEnvironmentReloading.kt:328)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$launchModuleByName(ApplicationEngineEnvironmentReloading.kt:32)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:316)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:307)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:335)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:307)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:144)
        at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:274)
        at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:185)
        at com.gbaranski.ApplicationKt.main(Application.kt:10)
        at com.gbaranski.ApplicationKt.main(Application.kt)

The same code works just fine on AMD64 Linux.

[zaleslaw]

It will be great if somebody with M1 will help to reproduce the issue with ONNX Runtime (the same problem is known for TF Runtime)

[Craigacp]

The ORT jar on Maven Central doesn't support macOS aarch64, though it works just fine if you compile it from source on an M1 (I got it working in December 2020 shortly after I got an M1).

I believe the plan is to add support for macOS aarch64 in a future release but it's a bit constrained by the lack of build resources for that platform.

TF-java has a hack to get macOS aarch64 support, but it's not going to happen to TF 1.15, I don't think the native build supports that platform.

[zaleslaw]

@Craigacp thanks for the comment! It's important to know

[mbakgun]

Hi all,
A possible fix comes with onnxruntime version : 1.12.1
microsoft/onnxruntime#11054

Currently org.jetbrains.kotlinx:kotlin-deeplearning-onnx:0.4.0 uses 1.11.0

For now, you might consider adding dependency into your build system;
if you're using gradle-kotlin you can add following,

implementation("com.microsoft.onnxruntime:onnxruntime:1.12.1")