Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[v1.x] Save/Load Gluon Blocks & HybridBlocks #19565

Merged
merged 10 commits into from
Dec 15, 2020

Conversation

samskalicky
Copy link
Contributor

@samskalicky samskalicky commented Nov 19, 2020

Description

v1.x branch version of PoC for saving & loading Gluon Blocks & HybridBlocks from #19535. Version for master branch is #19564. This PR is fundamentally different due to the changes on the master branch for block names/UUIDs compared to v1.x

This PR adds 2 new APIs to Gluon Blocks:

save

Saves the model parameters and a json structure of the model that stores the names of blocks' children in order to later restore and match the saved parameters. This does not actually store the model. Users still need to recreate their model architecture using their Gluon code in Python.

It also stores the hybridized Symbol objects for HybridBlocks and associated settings so that it can be later reloaded.

load

Loads the model parameters and a json structure of the model. It restores the names of blocks' children in order to match the saved parameters.

Also restores hybridized Symbol objects in HybridBlocks so they do not need to be hybridized again. After calling the load API users can immediately run inference without needing to hybridize.

@mxnet-bot
Copy link

Hey @samskalicky , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [edge, sanity, centos-cpu, unix-gpu, miscellaneous, clang, website, centos-gpu, windows-gpu, windows-cpu, unix-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 19, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 20, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 20, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 20, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 20, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 20, 2020
@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-review PR is waiting for code review labels Dec 12, 2020
@lanking520 lanking520 added pr-awaiting-review PR is waiting for code review pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review pr-work-in-progress PR is still work in progress labels Dec 12, 2020
@samskalicky samskalicky added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-review PR is waiting for code review labels Dec 12, 2020
Copy link
Contributor

@leezu leezu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must not change load_json -> fromjson in the 1.x branch due to API compatibility

@samskalicky
Copy link
Contributor Author

Must not change load_json -> fromjson in the 1.x branch due to API compatibility

backed out the change

@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-merge Review and CI is complete. Ready to Merge labels Dec 14, 2020
Copy link
Contributor

@leezu leezu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @samskalicky

@lanking520 lanking520 added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 15, 2020
@samskalicky samskalicky merged commit b5fd18a into apache:v1.x Dec 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-merge Review and CI is complete. Ready to Merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants