-
Notifications
You must be signed in to change notification settings - Fork 557
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
omnibus actual concurrency and major refactor (#1530)
* add concurrency to config * more descriptive names for predict functions * don't cancel from signal handler if a loop is running. expose worker busy state to runner * move handle_event_stream to PredictionEventHandler * make setup and canceling work * keep track of multiple runner prediction tasks to make idempotent endpoint return the same result and fix tests somewhat * drop Runner._result, comments * move create_event_handler into PredictionEventHandler.__init__ * break out Path.validate into value_to_path and inline get_filename and File.validate * split out URLPath into BackwardsCompatibleDataURLTempFilePath and URLThatCanBeConvertedToPath with the download part of URLFile inlined * let's make DataURLTempFilePath also use convert and move value_to_path back to Path.validate * prediction->request * split up predict/inner/prediction_ctx into enter_predict/exit_predict/prediction_ctx/inner_async_predict/predict/good_predict as one way to do it. however, exposing all of those for runner predict enter/coro exit still sucks, but this is still an improvement * bigish change: inline predict_and_handle_errors * inline make_error_handler into setup * move runner.setup into runner.Runner.setup * add concurrency to config in go * try explicitly using prediction_ctx __enter__ and __exit__ * relax setup argument requirement to str * glom worker into runner * add logging message * fix prediction retry and improve logging * split out handle_event * use CURL_CA_BUNDLE for file upload * dubious upload fix * skip worker and webhook tests since those were erroring on removed imports. fix or xfail runner tests * validate prediction response to raise errors, but return the unvalidated output to avoid converting urls to File/Path * expose concurrency in healthcheck * mediocre logging that works like print * COG_DISABLE_CANCEL to ignore cancelations * COG_CONCURRENCY_OVERRIDE * add ready probe as an http route * encode webhooks only after knowing they will be sent, and bail our of upload type checks early for strs * don't validate outputs * add AsyncConcatenateIterator * should_exit is not actually used by http * format * codecov * describe the remaining problems with this PR and add comments about cancelation and validation * add a test * fix test (#1669) * fix config schema * allow setting both max and target concurrency in cog.yaml (#1672) * drop default_target (#1685) --------- Signed-off-by: technillogue <technillogue@gmail.com> Co-authored-by: Mattt <mattt@replicate.com>
- Loading branch information
1 parent
5fe5ba3
commit f963078
Showing
14 changed files
with
486 additions
and
361 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.