Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merging PRs from
canary
intomain
to releaseturnkeyml v1.0.0
.Closes #50
Closes #39
Closes #26
Closes #21
Closes #10
Closes #8
Closes #36
Closes #51
Closes #55
Closes #57
Version 1.0.0
This version focuses on cleaning up technical debts and most of the changes are not visible to users. It removes cumbersome requirements for developers, removes unused features to streamline the codebase, and also clarifying some API naming schemes.
Users, however, will enjoy improved fidelity in their reporting telemetry thanks to the streamlined code.
Users
User Improvements
Improvements to the information in
turnkey_stats.yaml
and report CSVs:build_status
andbenchmark_status
now accurately report the status of their respective toolchain phases.benchmark_status
was a superset of the status of both build and benchmark.User Breaking Changes
None.
Developers
Developer Improvements
SetSuccess
Stage is no longer required at the end of every Sequence.build_model()
would only return aState
object if thestate.build_status == successful_build
, which in turn had to be manually set in a Stage.state.build_status = successful_build
on your behalf.Developer Breaking Changes
benchmark_model()
API has been removed as there were no known users / use cases. Anyone who wants to run standalone benchmarking can still instantiate anyBaseRT
child class and callBaseRT.benchmark()
..txt
files in the cache have been removed since no code was using those APIs. Labels are now saved intoturnkey_stats.yaml
instead.quantization_samples
argument to thebuild_model()
API has been removed.Stats
has been adjusted for consistency. It used to refer to both builds and benchmarks as "builds", whereas now it uses "evaluations" as a superset of the two.Stats.add_build_stat()
is nowStats.save_model_eval_stat()
.Stats.add_build_sub_stat()
is nowStats.save_model_eval_sub_stat()
.Stats.stat_id
is nowStats.evaluation_id
.builds
section of the stats/reports is nowevaluations
.Stats.save_stat()
is nowStats.save_model_stat()
.Stats.build_stats
is nowStats.evaluation_stats
.SetSuccess
build stage has been removed because build success has been reworked (see improvements).logged_subprocess()
API has been moved from thecommon.build
module to therun.plugin_helpers
module.