[Documentation] Document TensorRT engine encryption dll specification

[BengtGustafsson]

Describe the documentation issue

As I understand there is now support for engine encryption but as far as I can tell there is no documentation for how to write an encryption dll. What its function's names should be, what parameters they have and their semantics.

We need this as there is no movement on the issues #22179 and #22165 which would be a better solution: Then we can keep our weights in our encrypted onnx file and provide it and the stripped engine file when creating a new Engine.

Another option which seems better than an encryption library is to provide a buffer-based API for engine caching. I guess this is somewhat problematic as the caller can't create a buffer beforehand as there is no way to know what size is required. But why not use some kind of stream API or just a simple callback like storeBytes(void*, size_t) that the EP can call when it has done its engine optimization (once or multiple times has to be defined). This way we can keep the engine data in RAM until we can encrypt it and then store it to disk. This has the advantage of not having to redistribute an extra dll which is easy to shim to get hold of the unencrypted data.

Yes now that I wrote this I feel that a dll/so to do encryption is a very bad idea. It provides the easiest possible way to get hold of the unencrypted data: Just make a tiny program that calls the function with the contents of the encrypted engine and write out the decrypted contents! Essentially no protection!

So to sum up: a) remove the dll support, or if not PLEASE DO NOT document the API to at least make it somewhat harder to abuse the dll to decrypt engines to steal their contents. b) make a in memory buffer possibility to handle engine file contents so we can encrypt it and store it as we like instead of being forced to store it in a disk file. I think this is what TensorRT itself does.

Page / URL

No response

[BengtGustafsson]

I also spoke with our quality manager and it seems that some requirements in ISO 13485 regarding the integrity of data pertaining software for medical equipment are hard to fulfil without proper encryption support. This is the reverse problem: with access to the encryption library an adversary can encrypt some phony engine file and replace the genuine one we're supplying.

While this can be counteracted towards end user equipment by delivering a dll that can only decrypt this is not possible for us as we don't know which exact GPU model users have so we have to do encryption after optimization. Delivering two separate encryption dlls to our OEM customers is just too complicated for them to handle.

[BengtGustafsson]

I checked your source code and in about 5 minutes I acquired enough knowledge to be able to decrypt any engine file that anyone out there has encrypted using this mechanism. The weakness is of course that users have to provide the decrypter in dll form so that any perpetrator can call the decrypt function in the dll to get the original unencrypted data stream. It would have been much better if you accepted a function pointer in the OrtTensorRTProviderOptionsV2 struct. Then a perpetrator would have had to reverse engineer the library using onnxruntime which is a much harder task than to write a 10 line decryption program calling the dll.

Such a system would be acceptable to us, although I think I would prefer being able to handle the engine data as a memory buffer so that we can store it in whatever form we like, with whatever encryption we like.