[microTVM]Add test for MLPerfTiny models #13978
Conversation
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
Generated by tvm-bot |
mehrdadh
force-pushed
the
micro/add_mlperftiny_models
branch
from
f820fe6
to
6e90b2d
Compare
| @@ -19,6 +19,7 @@ | |||
|
|
|||
| #include "api/submitter_implemented.h" | |||
|
|
|||
| #include <assert.h> | |||
There was a problem hiding this comment.
yeah, it is used in TVMBackendAllocWorkspace
| project = tvm.micro.project.generate_project_from_mlf( | ||
| template_project_path, workspace_dir / "project", model_tar_path, project_options | ||
| ) | ||
| project.build() |
There was a problem hiding this comment.
is any validation of either the prediction or the running time results done for either the host-driven or mlperftiny tests?
There was a problem hiding this comment.
nevermind, I just reread your summary, 😁, but maybe add docstring with that info...
There was a problem hiding this comment.
yeah, we are testing accuracy here. I have added that to the doc string
| @pytest.mark.skip_boards( | ||
| ["mps2_an521", "mps3_an547", "stm32f746g_disco", "nucleo_f746zg", "nrf5340dk_nrf5340_cpuapp"] | ||
| ) | ||
| def test_mlperftiny_models(platform, board, workspace_dir, serial_number, model_name, project_type): |
There was a problem hiding this comment.
add docstring for test (maybe note what it does and doesn't test)
| @@ -19,6 +19,7 @@ | |||
|
|
|||
| #include "api/submitter_implemented.h" | |||
|
|
|||
| #include <assert.h> | |||
There was a problem hiding this comment.
yeah, it is used in TVMBackendAllocWorkspace
| @pytest.mark.skip_boards( | ||
| ["mps2_an521", "mps3_an547", "stm32f746g_disco", "nucleo_f746zg", "nrf5340dk_nrf5340_cpuapp"] | ||
| ) | ||
| def test_mlperftiny_models(platform, board, workspace_dir, serial_number, model_name, project_type): |
| project = tvm.micro.project.generate_project_from_mlf( | ||
| template_project_path, workspace_dir / "project", model_tar_path, project_options | ||
| ) | ||
| project.build() |
There was a problem hiding this comment.
yeah, we are testing accuracy here. I have added that to the doc string
mehrdadh
force-pushed
the
micro/add_mlperftiny_models
branch
from
9012819
to
4ec52ba
Compare
* Add test for mlperftiny models for host_driven and EEMBC runner versions * refactor mlperftiny models in different links * fix link * address comments and fix error
This PR adds test for MLPerfTiny models for both
mlperftinyproject type andhost_driven. In host drive, a single input data is used for testing but inmlperftinycase we only build the project.