feat: change the logic to handle dynamic batching

[JoanFM]

This PR fixes the behavior of dynamic batching.

• It makes sure that no batch larger than preferred batch size gets into the Executor.
• It allows the requests to be returned to the client as they are processed without waiting for other requests to be ready, this should improve the average latency of the system.

[codecov]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (810d8d4) 76.75% compared to head (91a8a78) 76.87%.
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6066      +/-   ##
==========================================
+ Coverage   76.75%   76.87%   +0.12%     
==========================================
  Files         145      145              
  Lines       13925    13986      +61     
==========================================
+ Hits        10688    10752      +64     
+ Misses       3237     3234       -3     

Flag	Coverage Δ
jina	76.87% <95.83%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
jina/serve/runtimes/worker/batch_queue.py	96.59% <97.61%> (+0.99%)	⬆️
jina/serve/runtimes/worker/request_handling.py	81.36% <83.33%> (+0.17%)	⬆️

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JohannesMessner]

just one tiny comment

[JohannesMessner]

btw i think it is fair to call this a feature or at least a performance improvement, the old implementation was correct (we never made any guarantees that the actual batch size would not be bigger than the preferred one), just not optimized for this scenario.