Add Centralized Manager support

[joseph-v]

What this PR does / why we need it:
Add Centralized Manager support

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #549

Special notes for your reviewer:

Link to Design document: sodafoundation/architecture-analysis#120

This feature is verified using VMAX simulator.
Following is the steps to run VMAX Simulator in an Ubuntu 18.04 machine.

git clone https://github.com/joseph-v/delfin.git
cd delfin
git checkout vmax-simulator-cm
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
cd delfin/tests/simulator/dell_emc/
mkdir -p vmax/data
pip install flask_restful
python vmax/vmax.py

This will start a VMAX simulator on the http://localhost:8443

To Test Centralized Manager feature with VMAX Simulator:

1. Start the delfin processes of (api.py, task.py & alert.py) and register VMAX storage with following commands. Storages should be successfully registered and resource and metrics collection logs can be verified from VMAX simulator and Delfin processes

# First storage
curl --location --request POST 'http://127.0.0.1:8190/v1/storages' \
--header 'Content-Type: application/json' \
--data-raw '{
    "vendor":"dellemc",
    "model":"vmax",
    "storage_name": "000196700153",
    "rest": {
        "host": "127.0.0.1",
        "port":8443,
        "username":"admin",
        "password":"password"
    }
}'

# Second storage
curl --location --request POST 'http://127.0.0.1:8190/v1/storages' \
--header 'Content-Type: application/json' \
--data-raw '{
    "vendor":"dellemc",
    "model":"vmax",
    "storage_name": "000196700154",
    "rest": {
        "host": "127.0.0.1",
        "port":8443,
        "username":"admin",
        "password":"password"
    }
}'

2. Register VMAX with any other "storage_name" should result in error response
3. Register an already registered storage_name should result in error response

To Test Centralized Manager feature with Embedded array using fake driver:

1. Register fake driver without "storage_name" in access_info, should result in success

curl --location --request POST 'http://127.0.0.1:8190/v1/storages' \
--header 'Content-Type: application/json' \
--data-raw '{
    "vendor":"fake_storage",
    "model":"fake_driver",
    "rest": {
        "host": "127.0.0.1",
        "port":8000,
        "username":"admin",
        "password":"password"
    }
}'

2. Repeat above register request, should result in error response (StorageAlreadyExists error)
3. Repeat above register request by adding "storage_name", should result in success

To Test SNMP Alert using VMAX Simulator:

1. Add trap_receiver_port = 8162 to delfin.conf, under [DEFAULT]
2. Start delfin/cmd/alert.py --config-file=/etc/delfin/delfin.conf
3. Configure SNMP for the array, Eg

curl --location --request PUT 'http://127.0.0.1:8190/v1/storages/01c0cf16-b595-4364-b5fd-8a4752120b71/snmp-config' \
--header 'Content-Type: application/json' \
--data-raw '{
  "version": "snmpv2c",
  "community_string": "public",
  "port": 8162,
  "host": "127.0.0.1"
}'

7. Send alert to above storage using following python script, snmptest.py using (command: python snmptest.py 000196700153)

import sys

from pysnmp.hlapi import *

mib = {
    ObjectIdentifier('1.3.6.1.3.94.1.11.1.3'): 123,
    ObjectIdentifier('1.3.6.1.4.1.1139.3.8888.2.0'): 'emcAsyncEventCode'
}


def main(argv):

    storage_id = argv[1]
    if not storage_id:
        storage_id = '000196700153'
    print("Sending SNMP alert to storage: ", storage_id)
    iterator = sendNotification(
        SnmpEngine(),
        CommunityData('public', mpModel=0),
        UdpTransportTarget(('127.0.0.1', 8162)),
        ContextData(),
        'trap',
        [
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.3'), OctetString('connUnitEventId')),
            ObjectType(ObjectIdentity('1.3.6.1.4.1.1139.3.8888.1.0.0'), OctetString('emcAsyncEventSource')),
            ObjectType(ObjectIdentity('1.3.6.1.4.1.1139.3.8888.2.0.0'), OctetString('emcAsyncEventCode')),
            ObjectType(ObjectIdentity('1.3.6.1.4.1.1139.3.8888.3.0.0'), OctetString('emcAsyncEventComponentType')),
            ObjectType(ObjectIdentity('1.3.6.1.4.1.1139.3.8888.4.0.0'), OctetString('emcAsyncEventComponentName')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.3.0'), OctetString('connUnitEventId')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.6.0'), OctetString('connUnitEventSeverity')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.7.0'), OctetString('connUnitEventType')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.8.0'), OctetString('connUnitEventObject')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.11.1.9.0'), OctetString('connUnitEventDescr')),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.6.1.20.0'), OctetString(storage_id)),
            ObjectType(ObjectIdentity('1.3.6.1.3.94.1.6.1.3.0'), OctetString('connUnitType')),
        ]
    )

    errorIndication, errorStatus, errorIndex, varBinds = next(iterator)

    if errorIndication:
        print(errorIndication)
    print("SNMP Alert Successfully send! ")


if __name__ == '__main__':
    main(sys.argv)

9. Check logs from alert process, that SNMP Trap is received by alert process and dispatched

Release note:

This feature is verified only with VMAX simulator and fake driver, and not verified in Hardware

[codecov]

Codecov Report

Merging #850 (c0ab246) into master (a44794d) will decrease coverage by 0.08%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master     #850      +/-   ##
==========================================
- Coverage   71.35%   71.26%   -0.09%     
==========================================
  Files         182      182              
  Lines       21240    21326      +86     
  Branches     3241     3263      +22     
==========================================
+ Hits        15156    15199      +43     
- Misses       5059     5095      +36     
- Partials     1025     1032       +7     

Impacted Files	Coverage Δ
delfin/alert_manager/trap_receiver.py	76.47% <ø> (+1.20%)	⬆️
delfin/api/schemas/storages.py	100.00% <ø> (ø)
delfin/task_manager/manager.py	0.00% <0.00%> (ø)
delfin/drivers/manager.py	53.75% <11.76%> (-11.33%)	⬇️
delfin/alert_manager/alert_processor.py	67.74% <34.61%> (-24.15%)	⬇️
delfin/api/v1/access_info.py	84.21% <60.86%> (-15.79%)	⬇️
delfin/drivers/driver.py	62.50% <75.00%> (+0.73%)	⬆️
delfin/api/v1/storages.py	75.62% <80.00%> (-0.41%)	⬇️
delfin/context.py	75.00% <100.00%> (+0.49%)	⬆️
delfin/db/sqlalchemy/models.py	99.66% <100.00%> (+<0.01%)	⬆️
... and 2 more

[wisererik]

LGTM

[skdwriting]

LGTM