Refactor model pruning framework #2504

chicm-ms · 2020-05-28T11:33:38Z

Before refactor:

A pruner class/instance includes 2 parts:

pruning algorithms implementation ( how to prune a specified weight with specified sparsity)
facilities to support the algorithms implementation:
- module wrapper
- config data and config validation
- hooks for optimizer.step()
- nn.Module forward hook for collecting activations (layer output)

problem: In one Pruner, it is not convenient to reuse another Pruner's pruning algorithm, especially for the algorithms need some intialization such as TaylorOF and APoz.

After refactor

Pruner interface for user is not changed, create a new pruning algorithms interface WeightMasker for code reuse
A previous pruner class is split into 2 classes:

A Masker class implements the pruning algorithms, with a common Masker interface.
A Pruner class is responsible for the facilities, and integrate a Masker class using general Masker interface, therefore it is easy to reuse any Masker class in any Pruner.

pull code

Fix gp tuner (microsoft#1592)

pull code

Fix compressor op_types (microsoft#1670)

pull code

Filter prune algo implementation (microsoft#1655)

pull code

document the dispatcher working dir (microsoft#1866)

src/sdk/pynni/nni/compression/torch/pruning/finegrained_prunning.py

src/sdk/pynni/nni/compression/torch/pruning/agp.py

QuanluZhang · 2020-06-10T15:06:39Z

@chicm-ms this refactor looks great. please update doc (i.e., tutorial) about how to write a new pruner.

chicm-ms · 2020-06-11T07:26:29Z

@chicm-ms this refactor looks great. please update doc (i.e., tutorial) about how to write a new pruner.

Thanks, the doc updated.

docs/en_US/Compressor/Framework.md

QuanluZhang · 2020-06-11T13:45:19Z

src/sdk/pynni/nni/compression/torch/pruning/finegrained_pruning.py

+            return {'weight_mask': torch.ones(weight.shape).type_as(weight)}
+        threshold = torch.topk(w_abs.view(-1), k, largest=False)[0].max()
+        mask_weight = torch.gt(w_abs, threshold).type_as(weight)
+        mask = {'weight_mask': mask_weight}


so bias has no mask in our implementation?

no, maybe add bias mask later?

docs/en_US/Compressor/Framework.md

ultmaster · 2020-06-11T13:27:45Z

docs/en_US/Compressor/Framework.md

 ```
-### Set wrapper attribute
-Sometimes `cal_mask` must save some state data, therefore users can use `set_wrappers_attribute` API to register attribute just like how buffers are registered in PyTorch modules. These buffers will be registered to `module wrapper`. Users can access these buffers through `module wrapper`.
+You can reference nni provided [weight masker](https://github.com/microsoft/nni/blob/master/src/sdk/pynni/nni/compression/torch/pruning/structured_pruning.py) implementations to implement your own weight masker.


Hard-coded link to source code is not a good idea. Recommend to use link to API docs instead. Still, keep it if you feel necessary.

Let's keep it for now.

docs/en_US/Compressor/Framework.md

ultmaster · 2020-06-11T13:37:56Z

src/sdk/pynni/nni/compression/torch/pruning/agp.py

+
+logger = logging.getLogger('torch pruner')
+
+class AGP_Pruner(Pruner):


Suggests AGPPruner

Maybe change it later, this PR is not intended to update interface.

ultmaster · 2020-06-11T13:38:34Z

src/sdk/pynni/nni/compression/torch/pruning/agp.py

+
+__all__ = ['AGP_Pruner']
+
+logger = logging.getLogger('torch pruner')


Suggest using package name. If torch pruner is actually used by all existing pruners, feel free to keep it.

OK, let's keep it for now.

src/sdk/pynni/nni/compression/torch/pruning/agp.py

src/sdk/pynni/nni/compression/torch/pruning/constants.py

ultmaster · 2020-06-11T13:42:06Z

src/sdk/pynni/nni/compression/torch/pruning/agp.py

+        assert span > 0
+        target_sparsity = (final_sparsity +
+                           (initial_sparsity - final_sparsity) *
+                           (1.0 - ((self.now_epoch - start_epoch) / span)) ** 3)


span could be 0 in default setting?

src/sdk/pynni/nni/compression/torch/pruning/one_shot.py

chicm-ms added 30 commits August 6, 2019 11:19

Merge pull request #31 from microsoft/master

3a45961

pull code

Merge pull request #32 from microsoft/master

633db43

pull code

Merge pull request #33 from microsoft/master

3e926f1

pull code

Merge pull request #34 from microsoft/master

f173789

pull code

Merge pull request #35 from microsoft/master

508850a

Fix gp tuner (microsoft#1592)

Merge pull request #36 from microsoft/master

5a0e9c9

pull code

Merge pull request #37 from microsoft/master

e7df061

pull code

Merge pull request #38 from microsoft/master

2175cef

pull code

Merge pull request #39 from microsoft/master

2ccbfbb

pull code

Merge pull request #40 from microsoft/master

b29cb0b

Fix compressor op_types (microsoft#1670)

Merge pull request #41 from microsoft/master

4a3ba83

pull code

Merge pull request #42 from microsoft/master

c8a1148

pull code

Merge pull request #43 from microsoft/master

73c6101

pull code

Merge pull request #44 from microsoft/master

6a518a9

pull code

Merge pull request #45 from microsoft/master

a0d587f

pull code

Merge pull request #46 from microsoft/master

e905bfe

pull code

Merge pull request #47 from microsoft/master

4b266f3

Filter prune algo implementation (microsoft#1655)

Merge pull request #48 from microsoft/master

237ff4b

pull code

Merge pull request #49 from microsoft/master

682be01

pull code

Merge pull request #50 from microsoft/master

133af82

pull code

Merge pull request #51 from microsoft/master

71a8a25

pull code

Merge pull request #52 from microsoft/master

d2a73bc

pull code

Merge pull request #53 from microsoft/master

198cf5e

pull code

Merge pull request #54 from microsoft/master

cdbfaf9

pull code

Merge pull request #55 from microsoft/master

7e9b29e

pull code

Merge pull request #56 from microsoft/master

d00c46d

pull code

Merge pull request #57 from microsoft/master

de7d1fa

pull code

Merge pull request #58 from microsoft/master

1835ab0

pull code

Merge pull request #59 from microsoft/master

24fead6

pull code

Merge pull request #60 from microsoft/master

0b7321e

document the dispatcher working dir (microsoft#1866)

updates

6573858

SparkSnail mentioned this pull request Jun 5, 2020

【Released】Iteration plan for June #2507

Closed

38 tasks

QuanluZhang requested review from liuzhe-lz, ultmaster, QuanluZhang and suiguoxin June 8, 2020 03:19

updates

fae7227

QuanluZhang reviewed Jun 10, 2020

View reviewed changes

src/sdk/pynni/nni/compression/torch/pruning/finegrained_prunning.py Outdated Show resolved Hide resolved

QuanluZhang reviewed Jun 10, 2020

View reviewed changes

src/sdk/pynni/nni/compression/torch/pruning/finegrained_prunning.py Outdated Show resolved Hide resolved

QuanluZhang reviewed Jun 10, 2020

View reviewed changes

src/sdk/pynni/nni/compression/torch/pruning/agp.py Show resolved Hide resolved

update docs

74c294f

chicm-ms added 3 commits June 11, 2020 15:28

updates

07d9b65

updates

981995d

update docs

8bb81c4

QuanluZhang reviewed Jun 11, 2020

View reviewed changes

docs/en_US/Compressor/Framework.md Outdated Show resolved Hide resolved

QuanluZhang reviewed Jun 11, 2020

View reviewed changes

docs/en_US/Compressor/Framework.md Outdated Show resolved Hide resolved

QuanluZhang reviewed Jun 11, 2020

View reviewed changes

ultmaster reviewed Jun 11, 2020

View reviewed changes

QuanluZhang reviewed Jun 11, 2020

View reviewed changes

src/sdk/pynni/nni/compression/torch/pruning/one_shot.py Outdated Show resolved Hide resolved

QuanluZhang approved these changes Jun 11, 2020

View reviewed changes

chicm-ms added 2 commits June 12, 2020 01:03

updates

d3405b6

updates

1ba2d7a

ultmaster approved these changes Jun 12, 2020

View reviewed changes

updates

ce3141b

chicm-ms merged commit 89fa23c into microsoft:master Jun 12, 2020

chicm-ms deleted the refactor-compressor-framework branch June 12, 2020 04:14

suiguoxin mentioned this pull request Jun 29, 2020

Auto pruners #2490

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor model pruning framework #2504

Refactor model pruning framework #2504

chicm-ms commented May 28, 2020 •

edited

Loading

QuanluZhang commented Jun 10, 2020

chicm-ms commented Jun 11, 2020

QuanluZhang Jun 11, 2020

chicm-ms Jun 11, 2020 •

edited

Loading

ultmaster Jun 11, 2020

chicm-ms Jun 12, 2020

ultmaster Jun 11, 2020

chicm-ms Jun 12, 2020

ultmaster Jun 11, 2020

chicm-ms Jun 12, 2020

ultmaster Jun 11, 2020


		logger = logging.getLogger('torch pruner')

		class AGP_Pruner(Pruner):


		__all__ = ['AGP_Pruner']

		logger = logging.getLogger('torch pruner')

Refactor model pruning framework #2504

Refactor model pruning framework #2504

Conversation

chicm-ms commented May 28, 2020 • edited Loading

Before refactor:

After refactor

QuanluZhang commented Jun 10, 2020

chicm-ms commented Jun 11, 2020

Choose a reason for hiding this comment

chicm-ms Jun 11, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chicm-ms commented May 28, 2020 •

edited

Loading

chicm-ms Jun 11, 2020 •

edited

Loading