Skip to content

Commit

Permalink
Merge pull request #3468 from mnogu/blobs_lr-to-lr_mult
Browse files Browse the repository at this point in the history
Replace blobs_lr with lr_mult in readme.md.
  • Loading branch information
longjon committed Dec 20, 2015
2 parents 7953918 + f19896c commit 03a84bf
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions examples/finetune_flickr_style/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ Because we are predicting 20 classes instead of a 1,000, we do need to change th
Therefore, we change the name of the last layer from `fc8` to `fc8_flickr` in our prototxt.
Since there is no layer named that in the `bvlc_reference_caffenet`, that layer will begin training with random weights.

We will also decrease the overall learning rate `base_lr` in the solver prototxt, but boost the `blobs_lr` on the newly introduced layer.
We will also decrease the overall learning rate `base_lr` in the solver prototxt, but boost the `lr_mult` on the newly introduced layer.
The idea is to have the rest of the model change very slowly with new data, but let the new layer learn fast.
Additionally, we set `stepsize` in the solver to a lower value than if we were training from scratch, since we're virtually far along in training and therefore want the learning rate to go down faster.
Note that we could also entirely prevent fine-tuning of all layers other than `fc8_flickr` by setting their `blobs_lr` to 0.
Note that we could also entirely prevent fine-tuning of all layers other than `fc8_flickr` by setting their `lr_mult` to 0.

## Procedure

Expand Down

0 comments on commit 03a84bf

Please sign in to comment.