After packing into JAR Windows doesn't register all engines, MacOS registers them

[filipsedivy]

Hello,

I would like to ask if you can guide me how to make sure that after packaging a JAR application, all the engines that are installed through the Maven dependency are registered?

When I run the app on macOS, this is what shows up in the console:

01:51:53.471 [Thread-0] DEBUG ai.djl.repository.zoo.ModelZoo - Searching model in specified model zoo: ai.djl.localmodelzoo
01:51:53.472 [Thread-0] DEBUG ai.djl.engine.Engine - Registering EngineProvider: TensorFlow
01:51:53.473 [Thread-0] DEBUG ai.djl.engine.Engine - Registering EngineProvider: OnnxRuntime
01:51:53.473 [Thread-0] DEBUG ai.djl.engine.Engine - Found default engine: TensorFlow
01:51:53.473 [Thread-0] DEBUG ai.djl.repository.zoo.ModelZoo - Checking ModelLoader: ai.djl.localmodelzoo:model.onnx UNDEFINED [
	ai.djl.localmodelzoo/model.onnx/model.onnx {}
]

When I run this same code under Windows, it does not register ONNX. Thus, the same code cannot be executed.

01:52:25.777 [Thread-3] DEBUG ai.djl.repository.zoo.ModelZoo - Searching model in specified model zoo: ai.djl.localmodelzoo
01:52:25.792 [Thread-3] DEBUG ai.djl.engine.Engine - Registering EngineProvider: TensorFlow
01:52:25.792 [Thread-3] DEBUG ai.djl.engine.Engine - Found default engine: TensorFlow
ai.djl.repository.zoo.ModelNotFoundException: ModelZoo doesn't support specified engine: OnnxRuntime

My maven is like this:

        <dependency>
            <groupId>ai.djl</groupId>
            <artifactId>api</artifactId>
            <version>0.16.0</version>
        </dependency>

        <dependency>
            <groupId>ai.djl.tensorflow</groupId>
            <artifactId>tensorflow-model-zoo</artifactId>
            <version>0.16.0</version>
        </dependency>

        <dependency>
            <groupId>ai.djl.tensorflow</groupId>
            <artifactId>tensorflow-engine</artifactId>
            <version>0.16.0</version>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>ai.djl.onnxruntime</groupId>
            <artifactId>onnxruntime-engine</artifactId>
            <version>0.16.0</version>
            <scope>runtime</scope>
        </dependency>

And here is part of the code I use to load the model

        Criteria<Image, Classifications> criteria = Criteria.builder()
                .setTypes(Image.class, Classifications.class)
                .optModelPath(path)
                .optTranslator(createTranslatorBuilder().build())
                .optEngine("OnnxRuntime")
                .build();

        ZooModel<Image, Classifications> model = criteria.loadModel();

Thank you very much for your advice.

[frankfliu]

It should just work on windows. A few question

• How you run your project in Windows?
• How you package your jar file?
• Can you share your project?

[filipsedivy]

I took the base code, and just put it into the console, and compiled that into a JAR package using IntelliJ tools (via Project Settings > Artifacts > Add JAR > ...)

And even though both TensorFlow and ONNXRuntime are in pom.xml, now ONNXRuntime is just registered for sure and TensorFlow is not.

13:57:27.779 [main] DEBUG ai.djl.repository.zoo.ModelZoo - Searching model in specified model zoo: ai.djl.localmodelzoo
13:57:27.779 [main] DEBUG ai.djl.engine.Engine - Registering EngineProvider: OnnxRuntime
13:57:27.779 [main] DEBUG ai.djl.engine.Engine - Found default engine: OnnxRuntime

I put the code here:
https://gist.github.com/filipsedivy/0ca5a8c0178e5936b3285fad0d68862d

[frankfliu]

@filipsedivy
I assume you created the jar with "From module with dependencies..." option. With this option, you can only add one engine into to jar.

DJL use ServiceLoader to locate all the engine, when creating fatjar, you have to merge SericeProvider file in the jar. See: #940

You can to use gradle or maven to create jar. I don't know if IntelliJ provides this advanced option. Here is example project to show you how to package a jar with DJL: ServiceLoader