Let bgp docker generate configuration by itself #173

taoyl-ms · 2016-12-29T23:18:21Z

A temp solution and call for comments.

Ideally python scripts and config.py should be put into docker-base or another layer of base image as the same logic will be shared by different dockers

lguohan · 2016-12-29T23:35:13Z

dockers/docker-bgp/Dockerfile

@@ -3,11 +3,15 @@ FROM docker-base
 COPY deps/quagga_*.deb /deps/
 RUN dpkg_apt() { [ -f $1 ] && { dpkg -i $1 || apt-get -y install -f; } || return 1; } &&        \
    dpkg_apt /deps/quagga_*.deb &&                                                              \
+    apt-get update &&                                                                           \
+    apt-get -y install -f python-jinja2 python-netaddr python-ipaddr python-lxml &&             \


move them to docker base as all the docker are going to use them.

Discussed with @lguohan @taoyl-ms, not to move to docker-base. Instead, package it and install in all the related docker images.

qiluo-msft · 2016-12-29T23:39:01Z

dockers/docker-bgp/bgpd.conf.j2

+! =========== Managed by Ansible DO NOT EDIT! ========================
+! generated by templates/quagga/bgpd.conf.j2 using minigraph_facts.py
+! file: bgpd.conf
+!


lguohan · 2016-12-29T23:47:46Z

dockers/docker-bgp/Dockerfile

    apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y &&                        \
    rm -rf /deps

 COPY daemons /etc/quagga/
+COPY ["*.j2", "config.sh", "template_list", "*.py", "/etc/swss/"]



can we embed the mountpoint here, e.g.,

VOLUME ['/etc/sonic']

Then, in the base image, we can put minigraph in the /etc/sonic/,

also, the VOLUME is in baseimage as it is shared by all dockers.

qiluo-msft · 2016-12-29T23:49:52Z

dockers/docker-bgp/minigraph_facts.py

@@ -0,0 +1,432 @@
+#!/usr/bin/env python


There are some concerns about

duplicate the minigraph parsers in every containers.

runtime overhead to generate config files on each bootup

The requirement is as:

there is no change of minigraph.xml after deployment

so, it is possible to create config files on host, copy to docker containers after they are just created. only the first bootup will be slower (if possible, we can run this step before reboot)

For 1, we are considering move those config-related scripts into docker-base after they are stabilized.

For 2, Guohan has some concern on potential changes of config file schema update container version update, thus we want to keep the capability of generating config file in the container itself.

Maybe Guohan could give more comments on this?

for 1, we can move it do the docker-base and stablized there.

for 2, configuration generate should be part of the docker container's job. For example, if we switch quagga to gobgp, the configuration template for them is going to be very different, where should be store the config template? I feel should be stored in the container.

minigraph_facts.py could be a python library that could be used by other config generators

I agree the configuration generation code should be inside each container.

qiluo-msft · 2016-12-29T23:54:54Z

dockers/docker-bgp/Dockerfile

    apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y &&                        \
    rm -rf /deps

 COPY daemons /etc/quagga/
+COPY ["*.j2", "config.sh", "template_list", "*.py", "/etc/swss/"]


etc [](start = 54, length = 3)

/etc is not a good place for scripts.

qiluo-msft · 2016-12-30T00:05:13Z

dockers/docker-bgp/render_config.py

+    data = parse_xml(minigraph)
+
+    env = jinja2.Environment(loader=jinja2.FileSystemLoader('./'), trim_blocks=True)
+    env.filters['ipv4'] = is_ipv4


filters [](start = 8, length = 7)

How many kinds of filters will we use? If many, we should consider implementation and maintenance cost. One option is to reuse ansible implementation.

ansible implementation is of GPLv3. There might be copyright concerns.

qiluo-msft · 2016-12-30T00:06:23Z

dockers/docker-bgp/config.sh

+
+cd /etc/swss
+
+awk '{system("python render_config.py -m /etc/swss/minigraph.xml "$1 ">"$2" && chown "$3" "$2" && chmod "$4" "$2)}' template_list 


Is this command too complex to read?

agree, use variable name would make it more readable.

qiluo-msft · 2016-12-30T00:10:47Z

dockers/docker-bgp/template_list

+bgpd.conf.j2  /etc/quagga/bgpd.conf    quagga:quaggavty  0644
+zebra.conf.j2 /etc/quagga/zebra.conf   root:root         0644
+isolate.j2    /usr/sbin/bgp-isolate    root:root         0755
+unisolate.j2  /usr/sbin/bgp-unisolate  root:root         0755


nisolate.j2 [](start = 1, length = 11)

suggest implement this feature as:

make fake config files in Dockerfile, set the correct ownership, groupship and mode. I guess the vanilla ones are already good.

copy file and maintain the target file attribute. ref: http://serverfault.com/questions/273949/cp-not-want-to-overwrite-permissions/273956

qiluo-msft

As comments.

lguohan · 2016-12-30T00:54:11Z

where is this file? https://github.com/Azure/sonic-mgmt/blob/master/ansible/roles/sonicv2/templates/quagga/daemons

lguohan · 2016-12-30T01:00:33Z

dockers/docker-bgp/Dockerfile

@@ -3,11 +3,15 @@ FROM docker-base
 COPY deps/quagga_*.deb /deps/
 RUN dpkg_apt() { [ -f $1 ] && { dpkg -i $1 || apt-get -y install -f; } || return 1; } &&        \
    dpkg_apt /deps/quagga_*.deb &&                                                              \
+    apt-get update &&                                                                           \
+    apt-get -y install -f python-jinja2 python-netaddr python-ipaddr python-lxml &&             \
    apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y &&                        \
    rm -rf /deps

 COPY daemons /etc/quagga/


daemons [](start = 5, length = 7)

where is this file?

it's already there. somebody add it in a previous version.
and, it's actually not a template

svc-acs · 2016-12-30T03:23:12Z

No test results found.

svc-acs · 2016-12-30T03:23:46Z

No test results found.

Signed-off-by: Shuotian Cheng <shuche@microsoft.com>

…-temp-access [platform/questone2bd] Remove SPD temperature sensors from platform

[marvell-armhf] Armada A385 soc support sonic-net#176 backport macsec xpn support to 4.19 kernel (sonic-net#174) add internal patch marker (sonic-net#173) Improve netfilter fullclone nat support patch sonic-net#171 Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>

[marvell-armhf] Armada A385 soc support #176 backport macsec xpn support to 4.19 kernel (#174) add internal patch marker (#173) Improve netfilter fullclone nat support patch #171 Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>

this PR updates the following commits 1673d25 [y_cable] refactor upgrade firmware API's; Fix vendor and part number API's read size for read_eeprom (sonic-net#174) ed93a15 [sonic_platform_base] Proper use of class and instance attributes (sonic-net#173) 691de92 [sonic_y_cable] add stub function for upgrade firmware of Y cable and split the get_part_number and get_vendor API's (sonic-net#171) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

Includes the following commits: 1673d25 [y_cable] refactor upgrade firmware API's; Fix vendor and part number API's read size for read_eeprom (#174) ed93a15 [sonic_platform_base] Proper use of class and instance attributes (#173) 691de92 [sonic_y_cable] add stub function for upgrade firmware of Y cable and split the get_part_number and get_vendor API's (#171) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

[muxcable] Fix Redis Selectable Processing (sonic-net#173) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

- [muxcable] Fix Redis Selectable Processing (#173) Signed-off-by: Danny Allen <daall@microsoft.com>

[muxcable] Fix Redis Selectable Processing (sonic-net#173) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

Includes the following commits: 1673d25 [y_cable] refactor upgrade firmware API's; Fix vendor and part number API's read size for read_eeprom (sonic-net#174) ed93a15 [sonic_platform_base] Proper use of class and instance attributes (sonic-net#173) 691de92 [sonic_y_cable] add stub function for upgrade firmware of Y cable and split the get_part_number and get_vendor API's (sonic-net#171) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

[muxcable] Fix Redis Selectable Processing (sonic-net#173) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

Selectable object can return multiple fields, however current code processes only the first field and wait till the next selectable is triggered. This PR consumes all fields that were returned by a selectable object. This results in reducing processing time from order of seconds to order of milliseconds. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>

Memory Histogram HLD

linkmgrd: * d2227d8 2023-02-22 | [active-standby] Toggle to standby if link down and config auto (sonic-net#173) (HEAD -> 202205) [Longxiang Lyu] Signed-off-by: Ying Xie <ying.xie@microsoft.com>

linkmgrd: * d2227d8 2023-02-22 | [active-standby] Toggle to standby if link down and config auto (#173) (HEAD -> 202205) [Longxiang Lyu] Signed-off-by: Ying Xie <ying.xie@microsoft.com>

Include commit: ``` 6ea1f03 Jing Zhang Tue Mar 28 08:42:44 2023 -0700 [202012] remove chatty log message for peer link event (#192) 198292d Jing Zhang Tue Mar 21 17:53:11 2023 -0700 [active-standby] avoid unnecessary mux state probe after configuring to `auto` (#183) 47de88e Jing Zhang Mon Mar 20 18:14:25 2023 -0700 [202012] Avoid unnecessary error logs from `handleGetServerMacAddressNotification` #96 (#185) 8a33319 Jing Zhang Mon Mar 6 11:53:27 2023 -0800 loose link down swithcover condition (#178) c2bf08d Jing Zhang Thu Mar 16 18:59:10 2023 -0700 fix ActiveStandbyStateMachine referrence (#186) 99d26af Jing Zhang Thu Mar 16 18:58:48 2023 -0700 [ci] Fix apt-get install unable locate package issue. (#177) (#187) d893be9 Longxiang Lyu Wed Feb 22 12:55:44 2023 +0800 [active-standby] Toggle to standby if link down and config auto (#173) ```

…nic-net#173) #### Description - Use class and instance attributes properly. Only use class attributes for data which should be shared among all instances. - Import all modules in `__init__.py` - Remove unused imports - Clean up whitespace #### Motivation and Context Proper object-oriented programming. This could prevent issues where attributes are shared between instances when they were not meant to be. This also fixes a bug with `PsuBase.psu_master_led_color`, where it is defined as both a class attribute and an instance attribute. With this change, some vendors' APIs can be cleaned up, because they redefined class attributes as instance attributes. That code can be removed. However, note that vendor APIs MUST call the base class initializer in their initializer methods, or the instance attributes will not be available and will raise exceptions.

all internal patches will be added below the marker. This eases internal patch management when public patches change Signed-off-by: Guohan Lu <lguohan@gmail.com>

…lly (#17436) #### Why I did it src/sonic-gnmi ``` * 88e82d4 - (HEAD -> master, origin/master, origin/HEAD) Replace PFC_WD_TABLE with PFC_WD (#173) (8 days ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#18104) #### Why I did it src/sonic-gnmi ``` * 50849ce - (HEAD -> 202305, origin/202305) Replace PFC_WD_TABLE with PFC_WD (#173) (22 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#18574) #### Why I did it src/sonic-gnmi ``` * 4b12af9 - (HEAD -> 202305, origin/202305) Merge pull request #209 from zbud-msft/revert_pfcwd_202305 (2 minutes ago) [StormLiangMS] * 802dbb7 - Revert "Replace PFC_WD_TABLE with PFC_WD (#173)" (28 hours ago) [Zain Budhwani] * e1397c4 - Revert "Account for GLOBAL key in PFC_WD (#178)" (28 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#18600) #### Why I did it src/sonic-gnmi ``` * cf52c74 - (HEAD -> 202311, origin/202311) Replace PFC_WD_TABLE with PFC_WD (#173) (21 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#18824) #### Why I did it src/sonic-gnmi ``` * 530c6bb - (HEAD -> 202311, origin/202311) Merge pull request #220 from sonic-net/revert-211-cherry/202311/173 (3 days ago) [Ying Xie] * db6f983 - (origin/revert-211-cherry/202311/173) Revert "Replace PFC_WD_TABLE with PFC_WD (#173)" (8 days ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#19731) #### Why I did it src/sonic-gnmi ``` * 4e6f5b1 - (HEAD -> 202405, origin/202405) Merge pull request #277 from zbud-msft/revert_pfcwd_202405 (3 hours ago) [bingwang-ms] * 547241a - Rerun pipeline (25 hours ago) [Zain Budhwani] * 9785246 - Revert "Replace PFC_WD_TABLE with PFC_WD (#173)" (25 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (#20158) #### Why I did it src/sonic-gnmi ``` * 95f4400 - (HEAD -> master, origin/master, origin/HEAD) Revert "Replace PFC_WD_TABLE with PFC_WD (#173)" (#284) (2 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

…lly (sonic-net#20158) #### Why I did it src/sonic-gnmi ``` * 95f4400 - (HEAD -> master, origin/master, origin/HEAD) Revert "Replace PFC_WD_TABLE with PFC_WD (sonic-net#173)" (sonic-net#284) (2 hours ago) [Zain Budhwani] ``` #### How I did it #### How to verify it #### Description for the changelog

taoyl-ms requested review from lguohan and qiluo-msft December 29, 2016 23:19

lguohan reviewed Dec 29, 2016

View reviewed changes

qiluo-msft reviewed Dec 29, 2016

View reviewed changes

Let bgp docker generate configuration by itself

fe497a9

taoyl-ms force-pushed the taoyl/config branch from 1aad0a3 to fe497a9 Compare December 29, 2016 23:40

lguohan reviewed Dec 29, 2016

View reviewed changes

qiluo-msft reviewed Dec 29, 2016

View reviewed changes

qiluo-msft reviewed Dec 30, 2016

View reviewed changes

qiluo-msft requested changes Dec 30, 2016

View reviewed changes

lguohan reviewed Dec 30, 2016

View reviewed changes

taoyl-ms closed this Jan 3, 2017

madhanmellanox pushed a commit to madhanmellanox/sonic-buildimage that referenced this pull request Mar 23, 2020

[swssconfig]: Removing obsolete configuration files (sonic-net#173)

2beee2e

Signed-off-by: Shuotian Cheng <shuche@microsoft.com>

monaliusa pushed a commit to monaliusa/sonic-buildimage that referenced this pull request Aug 7, 2020

Merge pull request sonic-net#173 from pphuchar/201911_q2bd-remove-SPD…

9e03c2a

…-temp-access [platform/questone2bd] Remove SPD temperature sensors from platform

dmytroxshevchuk pushed a commit to dmytroxshevchuk/sonic-buildimage that referenced this pull request Aug 31, 2020

Add packaging for syncd-rpc (sonic-net#173)

230f322

Sabareesh-Kumar-Anandan mentioned this pull request Dec 18, 2020

[sonic-linux-kernle] submodule update #6249

Merged

3 tasks

jleveque mentioned this pull request Mar 5, 2021

[sonic-platform-common] Update submodule #6974

Closed

vdahiya12 mentioned this pull request Mar 7, 2021

[sonic-platform-common] submodule update #6983

Merged

4 tasks

vdahiya12 added a commit to vdahiya12/sonic-buildimage that referenced this pull request Apr 16, 2021

[sonic-platform-daemons] submodule update

6eb6685

[muxcable] Fix Redis Selectable Processing (sonic-net#173) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

daall added a commit that referenced this pull request Apr 19, 2021

[202012][submodule] Update sonic-platform-daemons submodule (#7357)

f9d7553

- [muxcable] Fix Redis Selectable Processing (#173) Signed-off-by: Danny Allen <daall@microsoft.com>

yozhao101 mentioned this pull request Jul 19, 2021

[autorestart] teamd autorestart test fails postcheck #8095

Open

carl-nokia pushed a commit to carl-nokia/sonic-buildimage that referenced this pull request Aug 7, 2021

[sonic-platform-daemons] submodule update (sonic-net#7351)

6fd0bb1

[muxcable] Fix Redis Selectable Processing (sonic-net#173) Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

sg893052 pushed a commit to sg893052/sonic-buildimage that referenced this pull request Apr 25, 2022

Merge pull request sonic-net#173 from sg893052/MEM_HIST

c757f52

Memory Histogram HLD

vdahiya12 mentioned this pull request May 31, 2022

[sonic-platform-common][sonic-platform-daemons]submodule update #10993

Merged

6 tasks

yxieca mentioned this pull request Feb 22, 2023

[202205][linkmgrd] advance submodule head #13939

Merged

zjswhhh mentioned this pull request Mar 31, 2023

[202012][sonic-linkmgrd] submodule update #14480

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Let bgp docker generate configuration by itself #173

Let bgp docker generate configuration by itself #173

taoyl-ms commented Dec 29, 2016

lguohan Dec 29, 2016

qiluo-msft Jan 3, 2017

qiluo-msft Dec 29, 2016

lguohan Dec 29, 2016

qiluo-msft Dec 29, 2016

taoyl-ms Dec 29, 2016

lguohan Dec 30, 2016

stcheng Dec 30, 2016

qiluo-msft Jan 3, 2017

qiluo-msft Dec 29, 2016

qiluo-msft Dec 30, 2016 •

edited

Loading

taoyl-ms Dec 30, 2016

qiluo-msft Dec 30, 2016

lguohan Dec 30, 2016

qiluo-msft Dec 30, 2016

qiluo-msft left a comment

lguohan commented Dec 30, 2016

lguohan Dec 30, 2016

taoyl-ms Dec 30, 2016

svc-acs commented Dec 30, 2016

svc-acs commented Dec 30, 2016


		cd /etc/swss

		awk '{system("python render_config.py -m /etc/swss/minigraph.xml "$1 ">"$2" && chown "$3" "$2" && chmod "$4" "$2)}' template_list

Let bgp docker generate configuration by itself #173

Let bgp docker generate configuration by itself #173

Conversation

taoyl-ms commented Dec 29, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiluo-msft Dec 30, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiluo-msft left a comment

Choose a reason for hiding this comment

lguohan commented Dec 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svc-acs commented Dec 30, 2016

svc-acs commented Dec 30, 2016

qiluo-msft Dec 30, 2016 •

edited

Loading