Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v1.0.0 #65

Merged
merged 7 commits into from
Dec 6, 2023
Merged

Release v1.0.0 #65

merged 7 commits into from
Dec 6, 2023

Conversation

jeremyfowers
Copy link
Collaborator

@jeremyfowers jeremyfowers commented Dec 6, 2023

Merging PRs from canary into main to release turnkeyml v1.0.0.

Closes #50
Closes #39
Closes #26
Closes #21
Closes #10
Closes #8
Closes #36
Closes #51
Closes #55
Closes #57

Version 1.0.0

This version focuses on cleaning up technical debts and most of the changes are not visible to users. It removes cumbersome requirements for developers, removes unused features to streamline the codebase, and also clarifying some API naming schemes.

Users, however, will enjoy improved fidelity in their reporting telemetry thanks to the streamlined code.

Users

User Improvements

Improvements to the information in turnkey_stats.yaml and report CSVs:

  • Now reports all model labels. Including, but not limited to, the model's OSS license.
  • build_status and benchmark_status now accurately report the status of their respective toolchain phases.
    • Previously, benchmark_status was a superset of the status of both build and benchmark.

User Breaking Changes

None.

Developers

Developer Improvements

  • Build success has been conceptually reworked for Stages/Sequences such that the SetSuccess Stage is no longer required at the end of every Sequence.
    • Previously, build_model() would only return a State object if the state.build_status == successful_build, which in turn had to be manually set in a Stage.
    • Now, if a Sequence finishes then the underlying toolflow will automatically set state.build_status = successful_build on your behalf.

Developer Breaking Changes

  • The benchmark_model() API has been removed as there were no known users / use cases. Anyone who wants to run standalone benchmarking can still instantiate any BaseRT child class and call BaseRT.benchmark().
  • The APIs for saving and loading labels .txt files in the cache have been removed since no code was using those APIs. Labels are now saved into turnkey_stats.yaml instead.
  • The quantization_samples argument to the build_model() API has been removed.
  • The naming scheme of the members of Stats has been adjusted for consistency. It used to refer to both builds and benchmarks as "builds", whereas now it uses "evaluations" as a superset of the two.
    • Stats.add_build_stat() is now Stats.save_model_eval_stat().
    • Stats.add_build_sub_stat() is now Stats.save_model_eval_sub_stat().
    • Stats.stat_id is now Stats.evaluation_id.
    • The builds section of the stats/reports is now evaluations.
    • Stats.save_stat() is now Stats.save_model_stat().
    • Stats.build_stats is now Stats.evaluation_stats.
  • The SetSuccess build stage has been removed because build success has been reworked (see improvements).
  • The logged_subprocess() API has been moved from the common.build module to the run.plugin_helpers module.

jeremyfowers and others added 7 commits December 6, 2023 10:03
Remove the deprecated quantization tool

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
* Refactor build out of benchmarking

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
* Remove the SetSuccess stage (and the need for it)

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>

* Add a comment about deepcopy

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>

* Fix CI testing order

* Move large test to bottom

---------

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
Co-authored-by: Jeremy Fowers <jeremy.fowers@amd.com>
Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
* Add v1.0.0 release notes
* Add public API contract
* Rev version number to v1.0.0
* Move logged_subprocess() from build to plugin_helers

---------

Signed-off-by: Jeremy Fowers <jeremy.fowers@amd.com>
Copy link
Collaborator

@danielholanda danielholanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and working!

@jeremyfowers jeremyfowers merged commit 0df8c8e into main Dec 6, 2023
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment