-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CCCP pooling layer #498
CCCP pooling layer #498
Conversation
@mavenlin, it's cool that you finally open sourced the algorithm. Why didn't you link to this PR in your blog post on CCCP pooling? I heard from a colleague who has been collaborating with your team that you were not using it in models trained for production environment. Why? Is it too slow compared to the vanilla convolution? |
@kloudkl I've opensourced it for quite a while in my own fork of caffe and cuda-convnet. |
4278286
to
c01f07a
Compare
@shelhamer CCCP is ready to be merged |
Great! Please format your model gist for the model zoo as done in Sergey's https://gist.github.com/sergeyk/034c6ac3865563b69e60 It should have a readme.md with commit + gist info, a solver prototxt, and On Saturday, September 13, 2014, Lin Min notifications@github.com wrote:
Evan Shelhamer |
@shelhamer It seems gist would put the files in alphabetical order. That's why my readme.md file is put after deploy.prototxt. I remove the deploy file and now it works. |
@mavenlin rather than introduce a whole new layer for this special case of convolution I have included an optimization in the Caffe convolution layer #1118. Once it is merged, please update your NIN definition to use CONV layers instead of CCCP, although of course you can keep the layer names to make their purpose clear. You can include the commit ID in the front matter then too. Thanks for the inaugural contribution to the model zoo! |
@shelhamer this is reasonable. BTW, I wonder if cudnn can do better in this, if the num dimension is also paralleled. |
@mavenlin the num dimension is parallelized in our cuDNN integration. Once Please close the PR once the model's updated to signal it's ready for a On Saturday, September 20, 2014, Lin Min notifications@github.com wrote:
|
@mavenlin note that you can include the deploy.prototxt too if you just make the model zoo link include the anchor for the readme.md. That is, link to the gist file url instead of only the gist. |
@mavenlin please update your model gist to switch the CCCP type layers to CONV for BVLC/caffe compatibility. I know there is interest in using your model. Thanks! |
@shelhamer Sorry for leaving this pending for so long, I've updated the prototxt, I'm currently overseas, I'll update the model once I get back. |
model updated. |
Awesome! Thanks for contributing the Network-in-Network model. On Tuesday, September 30, 2014, Lin Min notifications@github.com wrote:
|
Actually, it is updated, but looks like not working... |
@ducha-aiki It takes some extra hacking to update the model weights. I have done this myself previously, and you may want to try my version https://drive.google.com/folderview?id=0B0IedYUunOQINEFtUi1QNWVhVVU&usp=drive_web |
@ronghanghu, thank you very much, I will try it. The funniest thing that I could not train it from the scratch not because of lack of GPU, but because of no free space for imagenet downloading :( |
Hi @mavenlin, I'm wondering what top-5 accuracy do you get on ImageNet with the NIN model ? With and without tests ? I've not read about it so far . Thanks. |
@emasa |
This pull request is to add a cascadable cross channel parametric(cccp) pooling layer.
The output feature maps of this layer is an parametric recombination of the input feature map.
It is used with relulayer on top of a convolutional layer. Each patch from the convolution input is mapped to its feature vector in the output feature map through a nonlinear function (multilayer perceptron).
The function is equivalent to 1x1 convolution. However, if convlayer is used for this purpose, there is unnecessary im2col operation.
The cccp layer is used to implement the idea in this paper: Network In Network, which achieved best performance on cifar-10 and cifar-100 datasets. As is shown here.
Using the long name cascadable cross channel parametric pooling is because I want the abbrev to be the same with Союз Советских Социалистических Республик, which is just cool.
Please don't merge yet, tests and examples will be added soon.