Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Yelov8 Translator optimization #2908

Merged
merged 3 commits into from
Dec 31, 2023
Merged

Conversation

gevant
Copy link
Contributor

@gevant gevant commented Dec 30, 2023

Description

Brief description of what this PR is about

  • Improved post-processing performance up to 40x by reducing expensive native calls

  • Additional argument 'maxBox' added to improve post-processing performance by reducing number of considered bounding boxes

  • Sunset file fixed,, previous version ignored first 4 rows, so recognized classes were 4 off. Adding 4 rows header fixes the problem. New headers pointing to ultralytics doc pages and original coco dataset page.

  • Change on yolov8_synset.txt is a backward incompatible, example sunset file was incorrectly formatted and predicted class categories were off by 4.

- Improved post-processing performance up to 40x by reducing expensive native calls
- Additional argument 'maxBox' added to improve post-processing performance by reducing number of considered bounding boxes
- Sunset file fixed,, previous version ignored first 4 rows, so recognized classes were 4 off. Adding 4 rows header fixes the problem. New headers pointing to ultralytics doc pages and original coco dataset page.
@gevant gevant requested review from zachgk, frankfliu and a team as code owners December 30, 2023 23:49
@@ -27,13 +29,17 @@
*/
public class YoloV8Translator extends YoloV5Translator {


final int maxBoxes;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage of final for members and parameters is generally discouraged in favor of readability.

See: https://docs.djl.ai/docs/development/development_guideline.html#coding-conventions

int sizeClasses = classes.size();
long sizeBoxes = features4OneImg.size(1);
ArrayList<IntermediateResult> intermediateResults = new ArrayList<>();
final NDArray rawResult = list.get(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid use final

@@ -65,6 +65,8 @@ public static DetectedObjects predict() throws IOException, ModelException, Tran
arguments.put("toTensor", true);
arguments.put("applyRatio", true);
arguments.put("threshold", 0.6f);
// for performance optimization maxBox parameter can reduce number of considered boxes from 8400
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We follow AOPS coding convention. Please reformat your code with the following command:

./gradlew fJ

- reformatted code
- removed final
@codecov-commenter
Copy link

codecov-commenter commented Dec 31, 2023

Codecov Report

Attention: 1333 lines in your changes are missing coverage. Please review.

Comparison is base (bb5073f) 72.08% compared to head (805c448) 72.56%.
Report is 940 commits behind head on master.

Files Patch % Lines
...va/ai/djl/modality/nlp/generate/TextGenerator.java 2.81% 276 Missing ⚠️
.../java/ai/djl/modality/nlp/generate/SeqBatcher.java 0.75% 132 Missing ⚠️
...ity/nlp/generate/ContrastiveSeqBatchScheduler.java 2.97% 98 Missing ⚠️
...i/djl/modality/nlp/generate/SeqBatchScheduler.java 9.83% 55 Missing ⚠️
.../java/ai/djl/modality/cv/BufferedImageFactory.java 40.96% 47 Missing and 2 partials ⚠️
...a/ai/djl/modality/nlp/generate/StepGeneration.java 2.04% 48 Missing ⚠️
api/src/main/java/ai/djl/ndarray/NDArray.java 43.42% 39 Missing and 4 partials ⚠️
...n/java/ai/djl/modality/cv/output/CategoryMask.java 22.00% 39 Missing ⚠️
...i/src/main/java/ai/djl/ndarray/NDArrayAdapter.java 71.21% 31 Missing and 7 partials ⚠️
.../cv/translator/SemanticSegmentationTranslator.java 37.50% 35 Missing ⚠️
... and 75 more

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2908      +/-   ##
============================================
+ Coverage     72.08%   72.56%   +0.47%     
- Complexity     5126     7203    +2077     
============================================
  Files           473      708     +235     
  Lines         21970    32022   +10052     
  Branches       2351     3337     +986     
============================================
+ Hits          15838    23238    +7400     
- Misses         4925     7189    +2264     
- Partials       1207     1595     +388     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

1. Fixes windows line return
2. Furthe reduce NDArray operations
3. Refactor example code
4. Add unittest for Yolov8Detection
@frankfliu frankfliu merged commit 0e6f143 into deepjavalibrary:master Dec 31, 2023
5 checks passed
frankfliu added a commit that referenced this pull request Apr 26, 2024
* Yelov8 Translator optimization

- Improved post-processing performance up to 40x by reducing expensive native calls
- Additional argument 'maxBox' added to improve post-processing performance by reducing number of considered bounding boxes
- Sunset file fixed,, previous version ignored first 4 rows, so recognized classes were 4 off. Adding 4 rows header fixes the problem. New headers pointing to ultralytics doc pages and original coco dataset page.

* addressed PR comments

- reformatted code
- removed final

* Refactor Yolov8 example

1. Fixes windows line return
2. Furthe reduce NDArray operations
3. Refactor example code
4. Add unittest for Yolov8Detection

---------

Co-authored-by: Frank Liu <frankfliu2000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants