CephFS Provisioner: Add secret namespace support.

[cofyc]

What this PR does / why we need it:

• Administrators can configure a dedicated secret namespace for CephFS provisioner (fix Ceph provisioner requires creating secrets in PVC namespaces #309)
• Fix a bug in deleting secrets (should be backward compatible)

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

fixes #309.

Special notes for reviewer:

We can deploy cephfs-provisioner in a dedicated namepsace, e.g. cephfs, and configure it to store secrets in this dedicated namespace. Then cephfs provisioner will don't need CREATE|DELETE permission on any namespace.

example deployment.yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cephfs-provisioner
  namespace: cephfs
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: cephfs-provisioner
    spec:
      containers:
      - name: cephfs-provisioner
        image: "quay.io/external_storage/cephfs-provisioner:latest"
        imagePullPolicy: IfNotPresent
        env:
        - name: PROVISIONER_NAME
          value: ceph.com/cephfs
        - name: PROVISIONER_SECRET_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        command:
        - "/usr/local/bin/cephfs-provisioner"
        args:
        - "-id=cephfs-provisioner-1"
        - "-v=4"
      serviceAccount: cephfs-provisioner

example role.yaml:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: cephfs-provisioner
  namespace: cephfs
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["create", "delete"]

example rolebinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: cephfs-provisioner
  namespace: cephfs
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cephfs-provisioner
subjects:
- kind: ServiceAccount
  name: cephfs-provisioner

cc @wongma7 @rootfs

[rootfs]

@jsafrane can you merge?

[rootfs]

ping @wongma7

[jsafrane]

sorry, I can't merge to this repo, "You're not authorized to merge this pull request." :-(