[tensorrt] Updates tensorrt to 9.2.0 #3091

frankfliu · 2024-04-15T22:05:40Z

Description

Brief description of what this PR is about

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here

--------- Co-authored-by: Administrator <Administrator@tech8> Co-authored-by: KexinFeng <fenkexin@amazon.com>

* Implement PtNDArraryEx.multiboxDetection * MultiboxDetection - code cleanup * MultiboxDetection - code cleanup * MultiboxDetection - code cleanup * MultiboxDetection - code cleanup * format code * Fix, add tests, and pass CI --------- Co-authored-by: Zach Kimberg <kimbergz@amazon.com>

…brary#2796) This reverts commit 3a90d0a.

This fixes the markdown headers to be h1 so they render correctly in docs.

…y#2818)

…valibrary#2806) * [api] Added Early stopping configuration (deepjavalibrary#38) * [api] Added Builder for Early stopping configuration (deepjavalibrary#38) * Explicitly set NDManager for dataset in EarlyStoppingListenerTest to make the test run on JDK11 in gradle.

This creates an abstraction for combining devices into a single device. The main use case for now is in DJL Serving TP_parallel. It will allow us to create a WorkerGroup and a PyPredictor for a set of devices and then track the usage of devices properly. It could also be used later for multi-gpu training or other multi-device cases.

…alibrary#2826)

* Updates doc versions to 0.24.0 Also moves android gradle.properties to the new 0.25.0. * Remove android change

* Updates XGBoost to 2.0.1 * Use devtools 8 * Updates based on new Xgboost JNI API. --------- Co-authored-by: Frank Liu <frankfliu2000@gmail.com>

Fixes deepjavalibrary#2840

* Added element-wise gauss error function (ERF) * Added element-wise arctan2 * Format java * Fixed docs * added * to other_ptr in Atan2

* Added 2D FFT * Format java * Add default fft2 * Convert array to vectors * Add inverse fft2 * Add better assersion in ifft2 test * Add really better assersion in ifft2 test * Move cast bellow ifft2 for unsupported exception * Format java * changed dims to axes * changed dims to axes

* only build triton binaries * install requests library * remove script

…javalibrary#2850)

Updates the navigation as a followup to deepjavalibrary/djl-serving#1316.

…avalibrary#3043)

* Increase DJL version to 0.27.0 * Update README

…alibrary#3053) Fixes deepjavalibrary#3048

…brary#3052) Fixes deepjavalibrary#3044

Avoid crash when token exceed model max length.

…ibrary#3067)

…3071)

Fixes: deepjavalibrary#3073

…deepjavalibrary#3087)

…avalibrary#3088) * [fix] Fix HfModelZoo and LlamaModelZoo concurrent init problem --------- Co-authored-by: Frank Liu <frankfliu2000@gmail.com>

…tor (deepjavalibrary#3083)

…brary#3084)

lanking520 · 2024-04-22T15:16:49Z

.github/workflows/native_jni_s3_tensorrt.yml

@@ -6,16 +6,11 @@ on:
 jobs:
  build-tensorrt-jni-linux:
    runs-on: ubuntu-latest
-    container: deepjavalibrary/ubuntu18.04:tensorrt-cuda116
+    container: deepjavalibrary/ubuntu20.04:tensorrt-cuda122


how about using 22.04?

SidneyLann and others added 30 commits September 19, 2023 17:36

To support Yolov8 (deepjavalibrary#2776)

950340f

--------- Co-authored-by: Administrator <Administrator@tech8> Co-authored-by: KexinFeng <fenkexin@amazon.com>

[onnxruntime] Upgrades OnnxRuntime to 1.16.0 (deepjavalibrary#2784)

da15713

build ft for sm90 (deepjavalibrary#2785)

15fd0d0

Updates LightGBM to 1.7.6 (deepjavalibrary#2793)

3a90d0a

Revert "Updates LightGBM to 1.7.6 (deepjavalibrary#2793)" (deepjavali…

8fd79db

…brary#2796) This reverts commit 3a90d0a.

[tokenizer] Allows import non-english model (deepjavalibrary#2797)

27c6a57

allow to just build for 1 flow (deepjavalibrary#2798)

d432a65

[api] Fixed NDList decode numpy file bug (deepjavalibrary#2804)

458933c

[api] Allows cancel Input (deepjavalibrary#2805)

2f4ebee

[ci] Fixes out of diskspace issue (deepjavalibrary#2808)

90059cd

[docs] Fixes markdown headers (deepjavalibrary#2812)

298ea1f

This fixes the markdown headers to be h1 so they render correctly in docs.

Bump up DJL version to 0.25.0 (deepjavalibrary#2809)

f0b4334

add gpu flag build for triton client (deepjavalibrary#2815)

fe86680

[xgb] Add .xgb file extension support (deepjavalibrary#2810)

1c5aef8

[tokenizers] Upgrade huggingface tokenizers to 1.14.1 (deepjavalibrar…

23e07cf

…y#2818)

[huggingface] Adds CrossEncoderTranslator (deepjavalibrary#2817)

85d9e85

Update README with release update (deepjavalibrary#2823)

7d68857

[api] Replace double-check singlton with lazy initialization (deepjav…

3927867

…alibrary#2826)

[api] Refactor PublisherBytesSupplier.java (deepjavalibrary#2831)

9b0c8c9

Updates doc versions to 0.24.0 (deepjavalibrary#2829)

6981d76

* Updates doc versions to 0.24.0 Also moves android gradle.properties to the new 0.25.0. * Remove android change

Updates XGBoost to 2.0.1 (deepjavalibrary#2833)

715e620

* Updates XGBoost to 2.0.1 * Use devtools 8 * Updates based on new Xgboost JNI API. --------- Co-authored-by: Frank Liu <frankfliu2000@gmail.com>

[tokenizer] Fixes tokenizer bug (deepjavalibrary#2843)

9f55189

Fixes deepjavalibrary#2840

Add erf and atan2 (deepjavalibrary#2842)

f84d3bb

* Added element-wise gauss error function (ERF) * Added element-wise arctan2 * Format java * Fixed docs * added * to other_ptr in Atan2

only build triton binaries (deepjavalibrary#2847)

8f6ff7c

* only build triton binaries * install requests library * remove script

[tokenizer] Update import script for huggingface_hub api change (deep…

e8ceef3

…javalibrary#2850)

[docs] Update serving configuration nav (deepjavalibrary#2853)

e315554

Updates the navigation as a followup to deepjavalibrary/djl-serving#1316.

frankfliu and others added 25 commits March 26, 2024 10:08

[pytorch] Allows to exclude certain DLL from pytorch directory (deepj…

091a41c

…avalibrary#3043)

Update checkstyle tool version to 10.14.2 (deepjavalibrary#3047)

5eb2b9e

Upgrade dependency version (deepjavalibrary#3049)

2c1e7fa

Increase DJL version to 0.27.0 (deepjavalibrary#3046)

a22360a

* Increase DJL version to 0.27.0 * Update README

Increase build version to 0.28.0 (deepjavalibrary#3050)

22b46a9

[llama.cpp] Fixes llama.cpp JNI build issue (deepjavalibrary#3057)

25bec4d

[docs] Update OnnxRuntime version in README. (deepjavalibrary#3056)

9dc1b40

[tokenizer] Upgrade rs-jni to 0.21.1 (deepjavalibrary#3051)

8283acd

[model-zoo] Avoid use properties as part of cached file path (deepjav…

ada3b98

…alibrary#3053) Fixes deepjavalibrary#3048

[tokenizer] Apply Dense layer for TextEmbeddingTranslator (deepjavali…

ae00167

…brary#3052) Fixes deepjavalibrary#3044

[model-zoo] Fixes mxnet yolo metadata.json (deepjavalibrary#3059)

f8791db

[pytorch] Updates PyTorch to 2.1.2 (deepjavalibrary#3070)

14e8f0f

[tokenizer] Add do_lower_case support (deepjavalibrary#3069)

2205a90

[tokenizer] set truncatation to default (deepjavalibrary#3068)

8bd3bf6

Avoid crash when token exceed model max length.

[tokenizer] set maxLength when importing huggingface model (deepjaval…

b35ad45

…ibrary#3067)

[tokenizer] Updates huggingface tokenizer to 0.15.2 (deepjavalibrary#…

fa9bfbd

…3071)

[api] Fixes ZipUtil bug when unzip large file (deepjavalibrary#3074)

412bf76

Fixes: deepjavalibrary#3073

[api] Fixes tarslip issue (deepjavalibrary#3075)

ce9884e

[docs] Updates javadoc link (deepjavalibrary#3077)

edd2e65

[docs][lmi] add new docs pages to doc site (deepjavalibrary#3081)

a920d21

[tokenizer] Adds option to disable sigmoid for CrossEncoderTranslator (…

6be4fd6

…deepjavalibrary#3087)

[fix] Fix HfModelZoo and LlamaModelZoo concurrent init problem (deepj…

4a619b1

…avalibrary#3088) * [fix] Fix HfModelZoo and LlamaModelZoo concurrent init problem --------- Co-authored-by: Frank Liu <frankfliu2000@gmail.com>

[api] Support "inputs" key in json request for text embedding transla…

4a434c2

…tor (deepjavalibrary#3083)

[api] Add batch support to TextEmbeddingServingTranslator (deepjavali…

3813405

…brary#3084)

[tensorrt] Updates tensorrt to 9.2.0

7839aef

frankfliu requested review from zachgk and a team as code owners April 15, 2024 22:05

xyang16 approved these changes Apr 16, 2024

View reviewed changes

lanking520 reviewed Apr 22, 2024

View reviewed changes

frankfliu force-pushed the master branch from ec89a66 to c68f8a7 Compare April 26, 2024 19:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[tensorrt] Updates tensorrt to 9.2.0 #3091

[tensorrt] Updates tensorrt to 9.2.0 #3091

frankfliu commented Apr 15, 2024

lanking520 Apr 22, 2024

[tensorrt] Updates tensorrt to 9.2.0 #3091

Are you sure you want to change the base?

[tensorrt] Updates tensorrt to 9.2.0 #3091

Conversation

frankfliu commented Apr 15, 2024

Description

lanking520 Apr 22, 2024

Choose a reason for hiding this comment