[configdb] Migrate minigraph configurations to DB #942

taoyl-ms · 2017-09-07T21:52:47Z

Modify minigraph parser output format so it fit DB schema
Modify configuration templates to fit new schema
Systemd services dependencies are modified so database starts before any configuration consumer

lguohan · 2017-09-08T15:31:36Z

build failure. #Resolved

taoyl-ms · 2017-09-08T15:36:30Z

This is depending on sonic-net/sonic-py-swsssdk#12. I'll need to merge that PR first and update submodule pointer to make this build.

In reply to: 328136365 [](ancestors = 328136365)

lguohan · 2017-09-08T15:40:23Z

src/sonic-config-engine/tests/test_cfggen.py

        output = self.run_script(argument)
-        self.assertEqual(output.strip(), "{'Ethernet116': {'name': 'ARISTA02T1', 'port': 'Ethernet1/1'}, 'Ethernet124': {'name': 'ARISTA04T1', 'port': 'Ethernet1/1'}, 'Ethernet112': {'name': 'ARISTA01T1', 'port': 'Ethernet1/1'}, 'Ethernet120': {'name': 'ARISTA03T1', 'port': 'Ethernet1/1'}}")
+        self.assertEqual(output.strip(), "{'mgmt_addr': None, 'hwsku': 'Arista', 'lo_addr': None, 'local_port': 'Ethernet112', 'type': 'LeafRouter', 'port': 'Ethernet1/1'}")

    def test_minigraph_peers_with_range(self):


test_minigraph_peers_with_range [](start = 8, length = 31)

I think we need to test every key/value in the minigraph here.
For example, the SYSLOG_SERVERS, NTP_SERVER,

lguohan · 2017-09-08T15:47:46Z

src/sonic-config-engine/minigraph.py

-    results['syslog_servers'] = syslog_servers
-    results['dhcp_servers'] = dhcp_servers
-    results['ntp_servers'] = ntp_servers
-    results['forced_mgmt_routes'] = mgmt_routes
    results['erspan_dst'] = erspan_dst


so, we are not converting this one? #Resolved

lguohan · 2017-09-08T15:48:25Z

src/sonic-config-engine/minigraph.py

    results['erspan_dst'] = erspan_dst
-    results['deployment_id'] = deployment_id
    results['ethernet_interfaces'] = ethernet_interfaces


is this one in config db? #Resolved

… Per VLAN) (#999) * [DHCP Relay]: Support new <DhcpRelays> minigraph tag; support multiple VLANs * Don't start dhcrelay in quiet mode so as to get startup output in syslog * Update sonic-cfggen tests to support new '<DhcpRelays>' tag * <DhcpRelays> tag is only present for VLANs which require a DHCP relay agent -- only parse if present * Don't attempt to configure a DHCP relay agent for VLANs without specified DHCP servers * Modify to work with Taoyu's minigraph/DB changes (#942) * Reduce number of DHCP servers in sonic-cfggen unit tests from 4 to 2 * Remove isc-dhcp-relay sample output file from sonic-cfggen test, as we no longer generate that file * Update Option 82 isc-dhcp-relay patch to load all interface name-alias maps into memory once at start instead of calling sonic-cfggen on each packet we relay * Remove executable permission from Jinja2 template * Set max hop count to 1 so that DHCP relay will only relay packets with a hop count of zero * Replace tabs with spaces * Modify overlooked sonic-cfggen call, use Config DB instead of minigraph * Also ensure > 1 VLAN requires a DHCP relay agent before outputting to template * Generate port name-alias map file using sonic-cfggen and parse that in lieu of parsing port_config.ini directly * No longer drop packets with hop count > 0; Instead, drop packets which already contain agent info

* msft_github/master: [snmpagent]: Update sonic-snmpagent submodule (sonic-net#1004) [swss]: Update the ipinip.j2 template to specify the IPv4 loopback address (sonic-net#998) Change all port_config.ini column headers from 'port' to 'index' (sonic-net#1001) [docker-platform-monitor]: Remove stale fancontrol.pid file (if exists) before starting fancontrol (sonic-net#1002) [swss]: Update swss/common submodules (sonic-net#988) [snmp]: Update py-swsssdk/snmpagent submodules (sonic-net#996) [Broadcom]: Update SAI version to 3.0.3.2-5 (sonic-net#997) [Mellanox]: Update outdated MSN2410 minigraph (sonic-net#995) Add time stamp suffix to the dirty images version string (sonic-net#958) [Mellanox]: Remove FW upgrade procedure in docker (sonic-net#989) [snmp]: Update sonic-snmpagent submodule (sonic-net#986) [Broadcom]: Update SAI version to 3.0.3.2-4 (sonic-net#983) [Ingrasys]: Add Ingrasys S8900-64XC sfputil function and update submodule (sonic-net#984) [mellanox]: Update Mellanox SAI version (sonic-net#979) [Ingrasys]: Change hwmon kernel modules installation order (sonic-net#980) [Makefile] Allowing interactive session with the slave docker-container (sonic-net#903) [snmp]: Update sonic-snmpagent submodule (sonic-net#978) Disable snmpd module disk_hw, so no syslog messages about unavailable disks (sonic-net#977) [teamd]: Remove deprecated blocking logic before starting teamd (sonic-net#976) [Broadcom]: Update libsai version to 3.0.3.2-2 (sonic-net#973) [device/accton]: Add a new supported device, AS7312-54X (sonic-net#955) [sairedis]: update sairedis submodule (sonic-net#974) lacp runner will send lacp update right after it received SIGINT (sonic-net#969) [config]: Fix management interface configuration (sonic-net#966) [Broadcom]: Update OpenNSL modules (sonic-net#970) [utilities]: Update sonic-utilities (sonic-net#968) [interfaces]: Change MTU value to 9100 (sonic-net#967) Framework to plugin Organization specific scripts during ONIE Image build (sonic-net#951) Always start with Forwarding State flag set for bgpd (sonic-net#963) Update sonic-utilities to be compatible with sonic-net#942 (sonic-net#965) [swss]: Fix the command to get HWSKU with sonic-cfggen (sonic-net#964) [bgp]: Fix the deployment_id with DEVICE_METADATA (sonic-net#962) [Ingrasys] Update Ingrasys submodule for S8900-54XC (sonic-net#954) [build/onie installer] Install grub for SONiC post migration from another NOS (sonic-net#949) [syncd]: Comment out unused docker-ptf-brcm.mk [Broadcom]: Update OpenNSL/SAI version (sonic-net#959) [swss]: Move swss/common/sairedis submodule to 201709 tag [sairedis]: update sairedis submodule head (sonic-net#956) [service template] Starting new docker when HWSKU change is detected (sonic-net#946) [config] Fix an issue that bgp asn data type is not consistent (sonic-net#953) [mellanox]: Update Mellanox SAI version ansd SDK version [Ingrasys] update port_config.ini and sfputil for ingrasys platforms (sonic-net#952) [frr]: RR client support in minigraph for FRR (sonic-net#923) [configdb] Migrate minigraph configurations to DB (sonic-net#942) [devices]: Add led plugin for Arista 7060CX-32S and 7260CX3-64 (sonic-net#945) [sonic-slave]: SLAVE_TAG should be for both Dockerfile and Dockerfile.user (sonic-net#950) [github]: add templates for submitting issues and PR (sonic-net#947) [rsyslog]: Use timegenerated instead of timestamp (sonic-net#944) [dell]: remove nos_to_sonic_grub.cfg (sonic-net#943) [slave.mk]: Apply series of patches to SONIC_PYTHON_STDEB_DEBS targets if they exist (sonic-net#941) [rsyslog]: Use SONiC template in containers (sonic-net#940) [Broadcom]: Remove BRCM_OPENNSL library and upgrade BRCM_SAI to 3.0.3.2 (sonic-net#938) [baseimage]: allocate varlog disk in the initramfs stage (sonic-net#936) [sairedis]: revert deadlock fix in sonic-sairedis submodule (sonic-net#934) Replace CRLF line endings with LF (sonic-net#932) Fix confusing comment (sonic-net#931) Update sfputil support for Ingrasys S9100 (sonic-net#929) [quagga]: Disable ipv4 over ipv6 and enable ipv6 over ipv4 peer group (sonic-net#922) [quagga] enable core dump for bgpd and zebra (sonic-net#927) [devices]: Update Dell s6100/z9100 platform modules (sonic-net#925) Revert "Migrate DEVICE_METADATA to db (sonic-net#919)" (sonic-net#928) Migrate DEVICE_METADATA to db (sonic-net#919) [devices]: Bump sonic-platform-modules-arista submodule (sonic-net#924) [image]: build sonic-broadcom.raw image for sonic conversion from ftos (sonic-net#901) [sonic-slave] Force pyangbind version to 0.5.10 (sonic-net#918) [Arista-7260CX3] Rename hwSKU Arista-7260CX3-64 to Arista-7260CX3-C64, introducing new hwSKU Arista-7260CX3-D108C8 (sonic-net#920) [devices]: modify sfputil plugins for mellanox devices for new platform API (sonic-net#916) [cavm]Update sai revision and packet driver (sonic-net#914) [translate-acl] Specify pyangbind version to not introduce new dependency (sonic-net#915) Update sfputil support for Arista platforms (sonic-net#912) Port speed (sonic-net#879) [Accton]: Add a new supported device AS5712-54X (sonic-net#898) [kernel]: update kernel submodule (sonic-net#910) [device]: Updated dell s6100 submodule to 5ab014 (sonic-net#909) [broadcom]: update broadcom sai package to 2.1.5.1-17 (sonic-net#908) IPv4 prefixes shouldn't be sent by default over IPv6 session with FRR. (sonic-net#905) [submodule]: update sonic linux kernel (sonic-net#906) [sonic-sairedis] update sairedis submodule (sonic-net#211, sonic-net#212) (sonic-net#904) [mlnx-fw-upgrade]: Define required FW version in build time. (sonic-net#902) [SAI]: Remove the SAI submodule from buildimage repo (sonic-net#893) [Submodule update]: sonic-utilities (sonic-net#888) Revert "[mellanox]: Update Mellanox SAI version" [mellanox]: Update Mellanox SAI version [utilities]: Update sonic-utilities submodule [swss-common]: Update sonic-swss-common submodule [mellanox]: Disable fsat boot mode for SX kernel [quagga]: Update sonic-quagga submodule [Broadcom]: Update Broadcom SAI/SDK version (sonic-net#883) Squash merge v1.0.3 branch onto master

[201911][thermal control] Backport changes from master branch (#929) [201911][config] Support abbreviation (#933) Add 'hw-management-generate-dump.sh' to 'show techsupport' command (#934) [fwutil]: Update fwutil to v2.0.0.0. (#942) Fixes bug for PFCWD feature parameters (#838) Fixed fast-reboot for BFN platform (#871) [config] Add 'interface transceiver' subgroup with 'lpmode' and 'reset' subcommands (#904) [warm-reboot]: added pre-check for ISSU file (#915) [config] Don't attempt to restart disabled services (#944)

…omatically (#20646) #### Why I did it src/sonic-swss-common ``` * 11e4055 - (HEAD -> master, origin/master, origin/HEAD) Add LOGGING table (#783) (2 days ago) [Yevhen Fastiuk] * f6c1614 - Add function ActionSchemaByNameAndObjectType to allow different ActionSchema formats. (#909) (5 days ago) [divyagayathri-hcl] * 1408db8 - Add more information when connect redis fail (#882) (5 days ago) [Junchao-Mellanox] * b686bb0 - Add helper function to validate interface name length (#931) (5 days ago) [Stepan Blyshchak] * dc75f23 - C-api: refine interface on Selectable classes (#945) (5 days ago) [erer1243] * acc4805 - Fix the ZMQ producer/consumer table does not update changes to the database issue. (#942) (6 days ago) [Hua Liu] * e812954 - Adding VRRP_TABLE name in APPL_DB (#927) (8 days ago) [vvbrcm] * bd0f2eb - Added new PORT_OPERR_TABLE table in STATE_DB (#941) (10 days ago) [Prince George] * ace2080 - Fix goext ut for bookworm (#940) (12 days ago) [ganglv] * c6364d1 - [common] add PerformanceTimer (#893) (12 days ago) [Yijiao Qin] * deee7d6 - add PIC_CONTEXT_TABLE to shcema.h (#919) (13 days ago) [Shuai Shang] * 0b5d994 - Fix unit test for goext (#938) (13 days ago) [ganglv] * 45d7cb0 - Initial implementation of C api (#915) (2 weeks ago) [erer1243] ``` #### How I did it #### How to verify it #### Description for the changelog

…omatically (sonic-net#20646) #### Why I did it src/sonic-swss-common ``` * 11e4055 - (HEAD -> master, origin/master, origin/HEAD) Add LOGGING table (sonic-net#783) (2 days ago) [Yevhen Fastiuk] * f6c1614 - Add function ActionSchemaByNameAndObjectType to allow different ActionSchema formats. (sonic-net#909) (5 days ago) [divyagayathri-hcl] * 1408db8 - Add more information when connect redis fail (sonic-net#882) (5 days ago) [Junchao-Mellanox] * b686bb0 - Add helper function to validate interface name length (sonic-net#931) (5 days ago) [Stepan Blyshchak] * dc75f23 - C-api: refine interface on Selectable classes (sonic-net#945) (5 days ago) [erer1243] * acc4805 - Fix the ZMQ producer/consumer table does not update changes to the database issue. (sonic-net#942) (6 days ago) [Hua Liu] * e812954 - Adding VRRP_TABLE name in APPL_DB (sonic-net#927) (8 days ago) [vvbrcm] * bd0f2eb - Added new PORT_OPERR_TABLE table in STATE_DB (sonic-net#941) (10 days ago) [Prince George] * ace2080 - Fix goext ut for bookworm (sonic-net#940) (12 days ago) [ganglv] * c6364d1 - [common] add PerformanceTimer (sonic-net#893) (12 days ago) [Yijiao Qin] * deee7d6 - add PIC_CONTEXT_TABLE to shcema.h (sonic-net#919) (13 days ago) [Shuai Shang] * 0b5d994 - Fix unit test for goext (sonic-net#938) (13 days ago) [ganglv] * 45d7cb0 - Initial implementation of C api (sonic-net#915) (2 weeks ago) [erer1243] ``` #### How I did it #### How to verify it #### Description for the changelog

Migrate minigraph configurations to DB

36e5643

taoyl-ms requested review from lguohan, stcheng and jleveque September 7, 2017 21:52

msftclas added the cla-not-required label Sep 7, 2017

Merge branch 'master' into taoyl/mini_migration

f2cf284

lguohan reviewed Sep 8, 2017

View reviewed changes

Update sonic-py-swsssdk pointer

b2553b6

jleveque added Enhancement ➕ Submodule Update ⏫ labels Sep 8, 2017

taoyl-ms added 2 commits September 8, 2017 22:40

Support port speed

6dc510a

Support MIRROR_SESSION table

08db470

taoyl-ms changed the title ~~Migrate minigraph configurations to DB~~ [configdb] Migrate minigraph configurations to DB Sep 8, 2017

Support ACL_TABLE table

acde708

lguohan approved these changes Sep 12, 2017

View reviewed changes

taoyl-ms merged commit c9cc7ae into sonic-net:master Sep 12, 2017

taoyl-ms added a commit to taoyl-ms/sonic-py-swsssdk that referenced this pull request Sep 13, 2017

Update schema according to sonic-net/sonic-buildimage#942

a9e2a9b

taoyl-ms mentioned this pull request Sep 19, 2017

Adapt to new minigraph_parser schema sonic-net/sonic-utilities#103

Merged

taoyl-ms added a commit to taoyl-ms/sonic-buildimage that referenced this pull request Sep 19, 2017

Update sonic-utilities to be compatible with sonic-net#942

8e7b998

taoyl-ms mentioned this pull request Sep 19, 2017

Update sonic-utilities to be compatible with #942 #965

Merged

taoyl-ms added a commit that referenced this pull request Sep 19, 2017

Update sonic-utilities to be compatible with #942 (#965)

5ba6f53

taoyl-ms added a commit that referenced this pull request Sep 20, 2017

Update sonic-utilities to be compatible with #942 (#965)

833ce1f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[configdb] Migrate minigraph configurations to DB #942

[configdb] Migrate minigraph configurations to DB #942

taoyl-ms commented Sep 7, 2017 •

edited

Loading

lguohan commented Sep 8, 2017 •

edited by taoyl-ms

Loading

taoyl-ms commented Sep 8, 2017

lguohan Sep 8, 2017

lguohan Sep 8, 2017 •

edited by taoyl-ms

Loading

lguohan Sep 8, 2017 •

edited by taoyl-ms

Loading

[configdb] Migrate minigraph configurations to DB #942

[configdb] Migrate minigraph configurations to DB #942

Conversation

taoyl-ms commented Sep 7, 2017 • edited Loading

lguohan commented Sep 8, 2017 • edited by taoyl-ms Loading

taoyl-ms commented Sep 8, 2017

lguohan Sep 8, 2017

Choose a reason for hiding this comment

lguohan Sep 8, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

lguohan Sep 8, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

taoyl-ms commented Sep 7, 2017 •

edited

Loading

lguohan commented Sep 8, 2017 •

edited by taoyl-ms

Loading

lguohan Sep 8, 2017 •

edited by taoyl-ms

Loading

lguohan Sep 8, 2017 •

edited by taoyl-ms

Loading