Bilinear initialization initializes all channels to be the same

[Eranpaz]

I'm pretty sure the bilinear initialization doesn't work as it should.
I've struggled with it for a while and finally tested only a deconv layer with bilinear initialization.

This is the prototxt:

name: "convLayer_example"
input: "data"
input_dim: 1
input_dim: 2
input_dim: 600
input_dim: 800
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 2
    kernel_size: 4
    stride: 1
    weight_filler {
      type: "bilinear"
    }
    bias_filler {
      type: "constant"

    }
  }
}

Then I'm running:

solver=caffe.SGDSolver('solver.prototxt')
solver.net.params['conv1'][0].data

array([[[[ 0.0625,  0.1875,  0.1875,  0.0625],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.0625,  0.1875,  0.1875,  0.0625]],

        [[ 0.0625,  0.1875,  0.1875,  0.0625],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.0625,  0.1875,  0.1875,  0.0625]]],


       [[[ 0.0625,  0.1875,  0.1875,  0.0625],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.0625,  0.1875,  0.1875,  0.0625]],

        [[ 0.0625,  0.1875,  0.1875,  0.0625],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.1875,  0.5625,  0.5625,  0.1875],
         [ 0.0625,  0.1875,  0.1875,  0.0625]]]], dtype=float32) 

Where to my understanding only the diagonal elements of the weights should be non zero and all other weights should be zero.

Am I missing something?

[shelhamer]

Refer to the original PR #2213. This filler is intended to be used with grouped convolution that makes it effectively diagonal as you outlined.

From https://github.com/BVLC/caffe/blob/master/CONTRIBUTING.md:

Please do not post usage, installation, or modeling questions, or other requests for help to Issues.
Use the caffe-users list instead. This helps developers maintain a clear, uncluttered, and efficient view of the state of Caffe.