update multi-phase doc #969
Conversation
QuanluZhang
requested review from
xuehui1991,
Crysple,
squirrelsc,
chicm-ms,
scarlett2018 and
leckie-chn
docs/en_US/multiPhase.md
Outdated
|
|
||
| Multi-phase experiments refer to experiments whose trial jobs request multiple hyper parameters from tuner and report multiple final results to NNI. | ||
| The above cases can be supported by the same feature, i.e., multi-phase execution. To support those cases, basically a trial job should be able to request multiple configurations from tuner. Tuner is aware of whether two configuration requests are from the same trial job or different ones. Also in multi-phase a trial job can report multiple final results. is there restriction on sequence??? |
There was a problem hiding this comment.
"is there restriction on sequence???" What is this?
docs/en_US/multiPhase.md
Outdated
|
|
||
| To use multi-phase experiment, please follow below steps: | ||
| Note that, currently multi-phase only supports one pattern of calling `nni.get_next_parameter()` and `nni.report_final_result()`: __call the former one, then call the later one; and repeat this pattern__. If calling `nni.get_next_parameter()` multiple times consecutively, calling `nni.report_final_result()` after that is reporting the final result of the latest retrieved configuration by `nni.get_next_parameter()`. |
There was a problem hiding this comment.
How about below?
Note that, nni.get_next_parameter() and nni.report_final_result() should be called sequentially. If nni.get_next_parameter() is called multiple times, and then nni.report_final_result() is called once, the result is associated to last configuration, which is retrieved from the last get_next_parameter call. So there is no result associated to previous get_next_parameter calls, and it may cause some multi-phases algorithm broken.
There was a problem hiding this comment.
thanks for the suggestion. updated
docs/en_US/multiPhase.md
Outdated
|
|
||
| 1. Implement nni.multi_phase.MultiPhaseTuner. For example, this [ENAS tuner](https://github.com/countif/enas_nni/blob/master/nni/examples/tuners/enas/nni_controller_ptb.py) is a multi-phase Tuner which implements nni.multi_phase.MultiPhaseTuner. While implementing your MultiPhaseTuner, you may want to use the trial_job_id parameter of generate_parameters method to generate hyper parameters for each trial job. | ||
| __Write trial code which leverages multi-phase:__ |
There was a problem hiding this comment.
How about add a section for implementation part like below. And explain it needs to implment both trial and tuner. I think it should explain tuner firstly, as it's core part.
Implement multi-phase experiment
There was a problem hiding this comment.
maybe i still do not make it clear. To use multi-phase, users do not have to implement a tuner. our built-in tuners all support multi-phase.
docs/en_US/multiPhase.md
Outdated
|
|
||
| Before writing a multi-phase tuner, we highly suggest you to go through [Customize Tuner](https://nni.readthedocs.io/en/latest/Customize_Tuner.html). Different from writing a normal tuner, your tuner needs to inherit from `MultiPhaseTuner` (in nni.multi_phase_tuner). The key difference between `Tuner` and `MultiPhaseTuner` is that the methods in MultiPhaseTuner are aware of additional information, that is, `trial_job_id`. With this information, the tuner could know which trial is requesting a configuration, and which trial is reporting results. This information provides enough flexibility for your tuner to deal with different trials and different phases. For example, you may want to use the trial_job_id parameter of generate_parameters method to generate hyperparameters for a specific trial job. | ||
|
|
||
| Of course, to use your multi-phase tuner, you should add `multiPhase: true` in your experiment yaml configure file. |
There was a problem hiding this comment.
It's already said in previous content, and should be a bullet also.
There was a problem hiding this comment.
Let's first simply highlight it :)
Make multi-phase doc more clear.